430.00 a
Description: Let's take a look at the detailed components of the LangChain framework.
#400#ML_Engineer_Basic#430#ML_Development_Tools#430.00#LangChain#430.00 a#Langchain_Components
PDFLoader
- Korean Alphabet Encoding
- special characters
- Speed
- Meta data (Impact to Output)
Choice -> PyPDFLoader (general)
TextSpliter
- A minimum devider standard
- chunk size and meaning in context
Recursive, HuggingFace, SemanticChuker (experimental)
Choice -> Lexical + Semantic (Ensemble)
Embedding
- Fit expression for each Docs
- Korean Alphabet Encoding
OpenAIEmbedding (easy, no pipeline for embedding, cost)
CacheBackedEmbeddings (text->hash ->key, namespace, fast)
10 times query ->OpenAIEmbedding 10 -> 10times cost
10 times query -> OpenAIEmbedding 1 * CacheBackedEmbeddings 9 -> 1times cost
Choice -> OpenAIEmbedding * CacheBackedEmbeddings
Vector Store
- Keyword Search vs Semantic Search
Semantic Search
cloud (Pinecone, Weaviate, ElasticSearch) and local (Chroma, FAISS)
Choice -> FAISS
Rerievers
- Multi query Retriever
- Most importat - less important - more important