There have been many blogs like this over the last years.
Yes, embeddings are computationally heavy, but they are not at all complicated and they provide a lot of benefit.
90% of "document" based RAG projects should view semantic search with embeddings as their primary method.
It's very powerful and so easy to implement that you could try it out and discover whether performance would be an issue rather than trying to anticipate it.
petesergeant · 2026-08-26 10:11:38 UTC
Embeddings are reasonably simple, but it’s a journey to get there, and I am very proud of the dog-heavy explainer I wrote on them: https://sgnt.ai/p/embeddings-explainer/
dizhn · 2026-08-26 10:17:36 UTC
This is very good. Thanks.
dotancohen · 2026-08-26 10:45:03 UTC
This is terrific, thank you! There's a typo in the following sentence:
> we don’t especially want to say that books on forestry and similar to books on puppies
^and^are
rglover · 2026-08-26 13:18:33 UTC
Started reading and will have to finish later but thank you for sharing. Very helpful post.
pantsforbirds · 2026-08-26 14:56:06 UTC
I think it's VERY project specific. If you are looking for anything technical at all, then keyword search almost always does better (in my experience). I'd actually recommend starting with keyword search, and then expanding with embeddings after you have a better idea of what your users are trying to determine.
aitchnyu · 2026-08-28 09:12:14 UTC
How is pgvector with Sentence Transformers, a CPU-only embedding model, compared to a model hosted by OpenAI?
nilirl · 2026-08-26 09:58:48 UTC
Maybe I'm old but where exactly are the "dragons"?
How is RAG any different from the search systems we've been building before LLMs? Is it the sudden need for everyone to design a search API and engine that's driven this trend?
If so, I'd like to see more design patterns around existing search problems:
- Correcting or backtracking based on feedback.
- Measuring relevance.
- Comparison with task-based pre-written queries. Does every LLM task need a full blown search engine? Why not a tightly scoped domain API for data retrieval?
TudorAndrei · 2026-08-26 10:03:25 UTC
It's just information retrieval packaged as something new.
kachnuv_ocasek · 2026-08-26 10:11:43 UTC
And you can't fundraise on some old "information retrieval".
mdp2021 · 2026-08-26 13:28:04 UTC
It's just information retrieval through a new NN based technology that allows to map concepts and ideas as the compression of long text into points in a multidimensional space that manages to compress even more dimensions than the given ones, through non-transparent engines that give different mappings and results, and still (the information retrieval) requires many more clever tricks than the simple idea of vector distance ordering because things do not quite work as they should.
Let's say it's just "computation packaged as something new". "Trivial things".
brabel · 2026-08-26 10:14:35 UTC
The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.
triangle · 2026-08-26 10:23:35 UTC
Vector embeddings predate LLMs. They have been used as far back as the early 2000s. They are a general machine learning technique, rather than LLM specific
ozim · 2026-08-26 12:21:28 UTC
Unfortunately LLMs made vector search more popular so it seems like something LLM specific.
What makes it worse, a lot of people in the thread equate vector search with RAG, whereas RAG is the name for anything that model can query so a user doesn't have to copy/paste feed it to the model manually like access to text files is RAG.
nilirl · 2026-08-26 10:28:20 UTC
Sure and that's a new technique for indexing and querying.
Where's the new design tension? Indexes always had to be monitored for freshness and queries have always needed cleaning or parsing.
ewidar · 2026-08-26 10:34:43 UTC
not really, vectorising text/books is old school ML by this point.
I wonder who was doing doing semantic search in the last century!
"The future is already here—It's just not very evenly distributed..."
vintermann · 2026-08-26 12:01:28 UTC
Sure, the idea of making a vector embedding for words, sentences, documents etc. is old, but the meat is in how you construct this embedding. I think embeddings have gotten quite a bit better since word2vec.
KaseyKim · 2026-08-26 11:12:29 UTC
right, it is the foundation of machine learning.
Angostura · 2026-08-26 10:05:04 UTC
I have a particular antipathy for articles too lazy to spell out acronyms on first use.
For those times you need to Red Amber Green your BM25
dotancohen · 2026-08-26 10:36:23 UTC
The audience for this piece is already very familiar with RAG. I don't want articles discussing e.g. OLED screens telling me what the acronym is - that would be a sign that the article is far below the level that I need.
vaylian · 2026-08-26 10:51:53 UTC
A hyperlink to Wikipedia would have solved that issue.
Lorean1 · 2026-08-26 11:06:55 UTC
Maybe if a person can't even google RAG they are not the intended audience of that article.
Zambyte · 2026-08-26 11:33:34 UTC
Eh, a healthy web is a web. I enjoy my preferred search engine, but surfing the web is becoming a lost medium.
tux3 · 2026-08-26 11:43:02 UTC
Hypermedia? In my hypertext markup language?
That is so not Web 5.0. Best I can offer is a support widget that pops up and keeps trying to talk to you until you interract with it.
ninkendo · 2026-08-26 11:42:41 UTC
When I hear stuff like this I always imagine going to a restaurant and asking the waitress for a menu and them replying “lol just google it”.
It’s not that I can’t or don’t know how, it’s rather that the expectation should be that a website should… link you to the information it believes to be relevant background. It’s why it’s called a “web”, linking is a core concept.
brazukadev · 2026-08-26 12:33:59 UTC
> When I hear stuff like this I always imagine going to a restaurant and asking the waitress for a menu and them replying “lol just google it”.
in this case there was a menu in the next empty table and you saw it but in place of getting it you want the waitress to get it for you. Which is a normal behavior but you could save your time by just getting the menu yourself.
AshleyGrant · 2026-08-26 13:05:48 UTC
No. It isn't. With acronyms, there's often plenty of potential things it can stand for, and if the person doesn't know enough to know which one is the correct acronym, Googling it isn't going to help them.
As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK."
The NWS knows this and automatically links to their glossary for both acronyms as well as jargon in their discussions. <-- See what I did there? What does NWS mean in this context? If only I had provided a link that would help you know. I very easily could have. I just didn't.
serf · 2026-08-26 15:38:53 UTC
>As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK."
it also serves as a minimum barrier to entry for the masses, which isn't always a bad thing.
if you're reading this stuff, and you can't figure out what kind of RAG that the search engine mentioned is being talked about through context clues, or you aren't clever enough to feed context into the search like 'hackers , computers, rag' as a query -- there is a very high probability that the person will have absolutely nothing constructive to add to the conversation that is about the topics they haven't even yet conceptualized or are aware of.
in that case that slight barrier to entry for the conversation will serve as a tool to produce less work for the moderators and derail less threads into uselessness.
(much like this stupid divergence.)
AshleyGrant · 2026-08-26 16:30:23 UTC
> it also serves as a minimum barrier to entry for the masses, which isn't always a bad thing.
No. Lowering the barrier of entry to those who are trying, in earnest, to learn about a new topic, to broaden their base of knowledge is NEVER a bad thing. None of us were born with the knowledge to read this (or similar) article. Trying to kick the ladder down after you have climbed it is terrible behavior and absolutely must be discouraged and stopped at all costs.
> much like this stupid divergence
The only thing stupid in this conversation is the insistence that folks who might know less than the author of the linked article or the poster do not deserve to have access to the information.
wldcordeiro · 2026-08-26 19:22:53 UTC
blah blah blah justifications for gatekeeping.
ninkendo · 2026-08-27 10:59:38 UTC
Given the title of the article is “RAG is simpler than you think”, and how I expected it to literally be an article explaining what RAG is and how it works, it’s more like going to the menu holder at the front of the restaurant, finding out it doesn’t have any menus in it, and then the waitress saying “we don’t have menus, dummy, just google it.”
serf · 2026-08-26 15:31:21 UTC
given the audience and the venue I think it's more like going to a restaurant as a customer and then asking the waiter to explain what a sandwich is.
ninkendo · 2026-08-27 10:52:40 UTC
I mean the article is entitled “RAG is simpler than you think”. I clicked the link thinking “I don’t know what RAG is, so ok I’m willing to learn something”, and the article… completely fails to explain it.
Given that the stated purpose of the article is to literally explain how simple something is, not explaining that thing seems a bit misleading, no?
I’m gonna get rich when I make a website explaining all the technical concepts in AI. Every article will just say “lol google it”, it’s gonna be great.
inigyou · 2026-08-26 13:02:08 UTC
I thought this would be a useless search that brought up pictures of rags, but indeed, DDG delivers a full page of results about retrieval-augmented generation for the query "rag"
mdp2021 · 2026-08-26 13:15:30 UTC
We can confirm, RAG has been a very big thing in the past few years. It's actually bewildering that it be new to some now - but we are also getting the vibe that some are living an ""AI"-nausea" that may be shielding them from some trends.
Angostura · 2026-08-26 15:02:16 UTC
I found the piece interesting, once I worked out what it was about. I strongly disagree that taking time to spell out acronyms should be taken as a signal that an article is low level.
arjie · 2026-08-26 15:51:45 UTC
For people familiar with the field, it would be like if you had every article about hardware read “Intel Central Processing Units (CPUs) with modern Double Data Rate 5 (DDR5) Random Access Memory (RAM) can be coupled with Nvidia Graphics Processing Units (GPUs) to run Large Language Models (LLMs) that are stored on Solid State Disks (SSDs)”. Just rapidly becomes unreadable.
The acronym constraint was valid in a pre-LLM world but now you are perhaps 3 clicks in a modern browser from learning.
If I read an article that spelled out Random Access Memory I would definitely treat that as a lay article.
triceratops · 2026-08-26 16:37:59 UTC
There's a middle-ground where you write out ambiguous acronyms ("rag" is an English word) and not unambiguous ones ("oled" only has one commonly-used meaning).
apavlinovic · 2026-08-26 10:06:12 UTC
The article sounds like AI slop with some predictable tells like short punctual sentences, bizarre jargon, and titles like "Recipe 4: On-The-Fly Embedding (The Fresh Data Play)"
Can we not reward junk like this? Most of the sentences are incomprehensible and provide zero actual argumentation, it's just a list of "whats" with no "whys"
dsego · 2026-08-26 10:28:56 UTC
You are right, now I noticed "Real talk" and "Why this is underrated" and I can't unsee it.
7734128 · 2026-08-26 11:54:09 UTC
They're absolutely right – and this is is why it's a load bearing observation that cuts to the heart of the issue.
khalic · 2026-08-26 10:13:24 UTC
> Why this is more flexible than embeddings
Oh boy...
refactor_master · 2026-08-26 10:19:03 UTC
Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls.
Then, throw it all in BigQuery. Handles all the vector stuff natively.
Sprinkle an agentic bot UI thing on top to make it appear all-knowing and magical.
I assume other vendors than Google have a similar batteries-included approach you can just plug in.
cpursley · 2026-08-26 10:28:26 UTC
Yep, lock into some vendor from day 1. Great idea!
orisho · 2026-08-26 10:30:20 UTC
Vendor lock in is 2025. Porting became trivial with LLMs advancing like they have.
cpursley · 2026-08-26 12:18:29 UTC
What I'm saying is pick transportable tech from day 1 so you can easily move if they shut down, hike prices, decide they don't like you, etc.
usernametaken29 · 2026-08-26 10:45:29 UTC
> embed everything the first time
This assumes your text is small. Try embedding pdf reports - though luck. It surely won’t fit into most embeddings. I can think of many more examples: books, news articles, medical reports, insurance claims etc. they’re all too big to “index it all at once”
robrorcroptrer · 2026-08-26 11:35:00 UTC
What about splitting bigger content into chunks before embedding?
freakynit · 2026-08-26 12:25:08 UTC
How are you gonna handle the relations that span across individual chunks... if a later chunk refers something from 2 chunks before using `it`, rather than proper name, how will you handle that? Because at query time, that later chunk would not match.
refactor_master · 2026-08-26 13:45:28 UTC
Humans usually have ways around that in longer documents eg page numbers, paragraphs, links.
If someone gave me a report, in my hands, that said “see ‘it’” I’d also be confused.
gf000 · 2026-08-26 13:56:18 UTC
Absolutely a novice in this topic, but I would imagine that by simply having sufficiently big chunks it's simply not a problem? You surely have enough information in like a couple of paragraphs to denote in vector space roughly what it is about. So that both chunks would get found by a vector search, and then whatever is the logic it may put the whole original text of those chunks into context, but in any case enough so that an LLM can "reason" about the references in-between the two.
harlanji · 2026-08-26 15:12:04 UTC
Chunks can only be as large as the embedding model’s token limit, about 512-1024 tokens usually. Anything longer gets truncated.
Natural language processing could expanded references, but it starts to get tricky. Do you use Graph RAG, embed another version of the chunk that is distinct from the full text version, etc.. Another layer of processing and data to keep in sync if the source dan be updated.
gf000 · 2026-08-26 16:16:42 UTC
(assuming English text)
512 tokens ≈ 350–400 words ≈ a long paragraph or two.
1024 tokens ≈ 700–800 words ≈ about a page and a half to two pages.
I would be very surprised if that amount of text is not enough to encode a general topic into the embedding (otherwise, what would be the whole point of them?).
So if there is a meaningful reference in C referring to A (assuming A-B-C consecutive 1-2 paragraphs), I would expect that the content of the two at least superficially resemble each other, and a vector query for one would return both. (And also, if I am including A in the context after retrieval, then I better give some context before-after as well -- and references tend to be local).
But feel free to prove me wrong, I'm mostly just guessing, not even an educated (in the given topic) guess here.
mdp2021 · 2026-08-26 13:20:28 UTC
What member freakynit said nearby about chunks and relations between chunks, plus the storage and information efficiency problem: make some calculations about storing vectors - for paragraphs and for collections of paragraphs -, then compare the needed space with the original data...
Because you could have clever ideas about vectors related to more paragraphs related in the document structure - but that would multiply the vectors. The index can become much bigger than the corpus.
khalic · 2026-08-26 12:27:18 UTC
you won't get anything out of a whole book embedding anyway, even a structured page is too much
j0selit0 · 2026-08-26 15:14:56 UTC
I'm sorry is this ironic or not? doesn't sounds simple at all
bob1029 · 2026-08-26 10:20:08 UTC
Agentic query rewrite on top of good old fashioned Lucene is the end game. This is effectively providing a lot of the same magic you get with the semantic approach. Allowing the agent to query the document store iteratively is where the capabilities become unbounded.
Embeddings and semantic search add non determinism on top of non determinism. This seems fundamentally cursed. Lexical is much easier to control, iterate and debug. The tools are incredibly mature. Your users will probably prefer it as well.
jankovicsandras · 2026-08-26 10:21:41 UTC
If someone has a Postgres db and want very simple RAG:
The repo includes also plpgsql_bm25rrf.sql : PL/pgSQL function for hybrid search ( plpgsql_bm25 + pgvector ) with Reciprocal Rank Fusion; and Jupyter notebook examples.
simianwords · 2026-08-26 10:22:39 UTC
OT but its interesting that none of the harnesses today use embeddings but just simple grep. I would not have predicted this
imtringued · 2026-08-26 10:40:02 UTC
Ok? I'm not seeing how that is interesting, you're exclusively focusing on coding which requires precise substring locations. Google is basically almost entirely driven by embedding models now.
simianwords · 2026-08-26 10:47:24 UTC
And why do you think coding didn’t benefit from embeddings? It was attempted many times and the industry gave up.
I find this interesting because practically no one is doing RAG on thier personal data which is something I wouldn’t have expected.
marginalia_nu · 2026-08-26 14:30:55 UTC
A lot of this is due the size of the corpus.
Grep falls apart for severely underspecified queries, which is the difficult part of web search. For any given query in web search there can be several millions of candidate results. You can get good results with FTS as well, but just finding phrase matches is inadequate, you need more ranking signals to find relevant results.
When Claude is looking for a function in your code base, it needs to sift through dozens of matches. This is not hard, and anything beyond grep is likely not worth the effort.
anthonypasq · 2026-08-26 16:25:40 UTC
cursor still uses embeddings and theyve found it works better than just grep
their data use policy from july 2026 explicitly mentions embeddings
usernametaken29 · 2026-08-26 10:41:20 UTC
I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably end up having to re-embed more or different chunks of your text to accommodate more and more precise embedding search - at which point you’ll go the last mile and do reranking etc etc all the while having to support the operational burden of vector search.
Then you turn around and build a search query with 500 keywords and sure it’s painful but it just works, accommodates all use cases, scales and is overall less annoying to maintain.
lacedeconstruct · 2026-08-26 10:47:05 UTC
I thought text search was always the first thing you try, then fuzzy search, then you go for RAG
ozim · 2026-08-26 12:08:02 UTC
I think Bitwarden implemented some vector search in their password search feature ... totally annoying it gives me back all kinds of stuff that I don't care.
I want fuzzy search like 95% of time and then I might consider having additional list of things that can be suggested by vector search.
a1o · 2026-08-26 12:21:56 UTC
A good UI could do these and also exact match, give some point system to the results, then order them and perhaps use a bold highlight to reflect what parts of the input query reflected in each result.
gwerbin · 2026-08-26 12:53:17 UTC
Bandcamp has had legendarily bad semantic search for as long as they've been around. It's often completely impossible to find an artist or album or song even when you type the exact name.
t_mahmood · 2026-08-26 15:18:47 UTC
ahh now I realize why I get so much completely irrelevant search results in many sites recently. I mean I'm searching for betel and you're giving me nuts. haha
itintheory · 2026-08-27 01:08:32 UTC
Bitwarden has lost the plot. The most recent Windows update is so bad. It has way lower information density in the UI, more buttons to click for the same use, no longer puts focus on the search field by default (this one makes me irrationally angry), and on one of my Win 11 installs can't lock the vault, manually or automatically. How could they mess up such a simple app that worked fine for so long?! What perverse incentives caused this nonsense?!
wongarsu · 2026-08-26 12:14:20 UTC
It's not like a simple embedding search takes that much longer to implement. Especially on short descriptions where you don't have to deal with chunking. And if you let an LLM write the code it's even less of a difference. Combine that with embedding search promising to solve all your search problems, and I understand why people often skip over full text search and go straight to embeddings
j0selit0 · 2026-08-26 15:20:31 UTC
I wish everyone thought like you, in my experience unfortunately it's not the case
EagnaIonat · 2026-08-26 15:31:41 UTC
Even that is an oversimplification unless you are doing something very basic.
Volume of documents, size of documents, versioning, frequency of update, documents similar or overlapping information, how much or exactly what you need for the LLM to understand, AI friendly documents, who has access and at what level, blue teaming, red teaming, multi-lingual, does the LLM know the domain language of the user and documents.
I probably missed a few things even with that.
kaon_2 · 2026-08-26 10:52:49 UTC
Can you elaborate? We have technicians searching in different languages. Also our knowledge base is often in different languages. I just don't see how full text search can work? Maybe in a problem space like a wiki where people always know what to search for?
jon-wood · 2026-08-26 11:04:48 UTC
Instinctively this feels like a two phase problem - start with some machine translation into a single spoken language and index that, then when people are querying do the same thing. When returning search results show them in the original language.
whilenot-dev · 2026-08-26 11:27:44 UTC
Why not create indexes for multiple languages, as that would also avoid double translation issues (e.g. GER [query] → ENG [index] → GER [document])?
j0selit0 · 2026-08-27 08:58:17 UTC
you would also need to maintain multiple indexes in multiple languages. I never had to do that - but I assume it's a pain
kaon_2 · 2026-08-26 11:29:09 UTC
Yes we've tried. It works. But jargon is hard. RAG with embeddings works all the same. The LLM doesn't mind receiving sources in Italian, french and German, and then outputting the answer in Japanese while providing the verbatim German jargon term in brackets
jameshart · 2026-08-26 12:03:41 UTC
Embedding search is effectively machine translation into a single common ‘language’ - embedding space - and then searching that; cleaner and less lossy than translating everything into English for searching, but harder to debug when it goes wrong.
tantalor · 2026-08-26 11:31:01 UTC
FTS like Elasticsearch supports cross-language (also called multi-language) search.
hnfong · 2026-08-26 18:55:34 UTC
Yes. Thank you for pointing this out.
I think there needs to be a linguist version of "what every programmer needs to know about (full?) text search"...
I'm not a linguist and I don't study languages, but I know enough to realize if a text search system is not designed for a particular language, it simply won't work. (As an example, to implement English search in a system for a hobby project, I had to import a US/UK spelling wordlist, and implement the Porter Stemming Algorithm. This is just for "one" language, and probably does not cover the other "English" dialects. Imagine doing a different workaround for every language in existence...)
RAG is actually a very language-agnostic way to work around those issues.
jameshart · 2026-08-26 12:14:55 UTC
I think people also overestimate the need for full text search when the one doing the querying is an LLM. If your underlying data is structured records, like a customer database, while humans might not have time or skills to figure out that when they want to search by phone number they need to do a join from the contacts table to the users table and normalize the phone number to look up first, making it best to just surface phone numbers as part of the data that is full/text-indexed… an agent is quite happy to handcraft the right SQL to find records that match on a specific field, given the right SKILLS.md and schema information. Turning fuzzy searches into exact DB lookups is a great way LLMs can augment users.
(Obviously this doesn’t apply to searching actual rich document data - for that, go all in on text search, embedding, etc)
clevergadget · 2026-08-26 12:20:09 UTC
I don't know what level of quality is required for this site but RAG is trash its just trash. its magic beans.
josh_p · 2026-08-26 12:33:26 UTC
I worked on getting an address database into elasticsearch years ago when it was still using modified tf-idf. Customers wanted FTS where a lot of the queries would be something like "100 First Ave, NY" or "200 2nd St, MN".
It was one of the most fun projects I've worked on in my career so far. I got a learn a lot about how US and international addresses worked, so many edge cases, and got to really understand how customers were using the existing search to make sure they weren't adding any duplicates to the database. Token filters and synonyms were neat and figuring out the right indexing strategy was a lot of fun.
It was a lot more work to get it right for most of the use-cases our customers had than just "throw it into ES and be done". That would probably have been fine for the 80/20 case, like you said, but I agree that the bulk of the work is going to be fine-tuning the search solution, whatever technology you're using.
oever · 2026-08-26 12:52:48 UTC
What's your opinion on nominatim? I find that it gives up quickly when there's one or two typos in an address. It nails your examples.
Comments
Yes, embeddings are computationally heavy, but they are not at all complicated and they provide a lot of benefit.
90% of "document" based RAG projects should view semantic search with embeddings as their primary method.
It's very powerful and so easy to implement that you could try it out and discover whether performance would be an issue rather than trying to anticipate it.
How is RAG any different from the search systems we've been building before LLMs? Is it the sudden need for everyone to design a search API and engine that's driven this trend?
If so, I'd like to see more design patterns around existing search problems:
- Correcting or backtracking based on feedback.
- Measuring relevance.
- Comparison with task-based pre-written queries. Does every LLM task need a full blown search engine? Why not a tightly scoped domain API for data retrieval?
Let's say it's just "computation packaged as something new". "Trivial things".
What makes it worse, a lot of people in the thread equate vector search with RAG, whereas RAG is the name for anything that model can query so a user doesn't have to copy/paste feed it to the model manually like access to text files is RAG.
Where's the new design tension? Indexes always had to be monitored for freshness and queries have always needed cleaning or parsing.
at least to me that seems the same as https://en.wikipedia.org/wiki/Word2vec for e.g.
"The future is already here—It's just not very evenly distributed..."
So: https://en.wikipedia.org/wiki/Retrieval-augmented_generation
That is so not Web 5.0. Best I can offer is a support widget that pops up and keeps trying to talk to you until you interract with it.
It’s not that I can’t or don’t know how, it’s rather that the expectation should be that a website should… link you to the information it believes to be relevant background. It’s why it’s called a “web”, linking is a core concept.
in this case there was a menu in the next empty table and you saw it but in place of getting it you want the waitress to get it for you. Which is a normal behavior but you could save your time by just getting the menu yourself.
As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK."
The NWS knows this and automatically links to their glossary for both acronyms as well as jargon in their discussions. <-- See what I did there? What does NWS mean in this context? If only I had provided a link that would help you know. I very easily could have. I just didn't.
it also serves as a minimum barrier to entry for the masses, which isn't always a bad thing.
if you're reading this stuff, and you can't figure out what kind of RAG that the search engine mentioned is being talked about through context clues, or you aren't clever enough to feed context into the search like 'hackers , computers, rag' as a query -- there is a very high probability that the person will have absolutely nothing constructive to add to the conversation that is about the topics they haven't even yet conceptualized or are aware of.
in that case that slight barrier to entry for the conversation will serve as a tool to produce less work for the moderators and derail less threads into uselessness.
(much like this stupid divergence.)
No. Lowering the barrier of entry to those who are trying, in earnest, to learn about a new topic, to broaden their base of knowledge is NEVER a bad thing. None of us were born with the knowledge to read this (or similar) article. Trying to kick the ladder down after you have climbed it is terrible behavior and absolutely must be discouraged and stopped at all costs.
> much like this stupid divergence
The only thing stupid in this conversation is the insistence that folks who might know less than the author of the linked article or the poster do not deserve to have access to the information.
Given that the stated purpose of the article is to literally explain how simple something is, not explaining that thing seems a bit misleading, no?
I’m gonna get rich when I make a website explaining all the technical concepts in AI. Every article will just say “lol google it”, it’s gonna be great.
The acronym constraint was valid in a pre-LLM world but now you are perhaps 3 clicks in a modern browser from learning.
If I read an article that spelled out Random Access Memory I would definitely treat that as a lay article.
Can we not reward junk like this? Most of the sentences are incomprehensible and provide zero actual argumentation, it's just a list of "whats" with no "whys"
Oh boy...
Then, throw it all in BigQuery. Handles all the vector stuff natively.
Sprinkle an agentic bot UI thing on top to make it appear all-knowing and magical.
I assume other vendors than Google have a similar batteries-included approach you can just plug in.
This assumes your text is small. Try embedding pdf reports - though luck. It surely won’t fit into most embeddings. I can think of many more examples: books, news articles, medical reports, insurance claims etc. they’re all too big to “index it all at once”
If someone gave me a report, in my hands, that said “see ‘it’” I’d also be confused.
Natural language processing could expanded references, but it starts to get tricky. Do you use Graph RAG, embed another version of the chunk that is distinct from the full text version, etc.. Another layer of processing and data to keep in sync if the source dan be updated.
512 tokens ≈ 350–400 words ≈ a long paragraph or two. 1024 tokens ≈ 700–800 words ≈ about a page and a half to two pages.
I would be very surprised if that amount of text is not enough to encode a general topic into the embedding (otherwise, what would be the whole point of them?).
So if there is a meaningful reference in C referring to A (assuming A-B-C consecutive 1-2 paragraphs), I would expect that the content of the two at least superficially resemble each other, and a vector query for one would return both. (And also, if I am including A in the context after retrieval, then I better give some context before-after as well -- and references tend to be local).
But feel free to prove me wrong, I'm mostly just guessing, not even an educated (in the given topic) guess here.
Because you could have clever ideas about vectors related to more paragraphs related in the document structure - but that would multiply the vectors. The index can become much bigger than the corpus.
Embeddings and semantic search add non determinism on top of non determinism. This seems fundamentally cursed. Lexical is much easier to control, iterate and debug. The tools are incredibly mature. Your users will probably prefer it as well.
https://github.com/jankovicsandras/plpgsql_bm25 BM25 search implemented in PL/pgSQL ( Unlicense / Public domain )
The repo includes also plpgsql_bm25rrf.sql : PL/pgSQL function for hybrid search ( plpgsql_bm25 + pgvector ) with Reciprocal Rank Fusion; and Jupyter notebook examples.
I find this interesting because practically no one is doing RAG on thier personal data which is something I wouldn’t have expected.
Grep falls apart for severely underspecified queries, which is the difficult part of web search. For any given query in web search there can be several millions of candidate results. You can get good results with FTS as well, but just finding phrase matches is inadequate, you need more ranking signals to find relevant results.
When Claude is looking for a function in your code base, it needs to sift through dozens of matches. This is not hard, and anything beyond grep is likely not worth the effort.
https://cursor.com/blog/semsearch
https://cursor.com/data-use
their data use policy from july 2026 explicitly mentions embeddings
I want fuzzy search like 95% of time and then I might consider having additional list of things that can be suggested by vector search.
Volume of documents, size of documents, versioning, frequency of update, documents similar or overlapping information, how much or exactly what you need for the LLM to understand, AI friendly documents, who has access and at what level, blue teaming, red teaming, multi-lingual, does the LLM know the domain language of the user and documents.
I probably missed a few things even with that.
I think there needs to be a linguist version of "what every programmer needs to know about (full?) text search"...
I'm not a linguist and I don't study languages, but I know enough to realize if a text search system is not designed for a particular language, it simply won't work. (As an example, to implement English search in a system for a hobby project, I had to import a US/UK spelling wordlist, and implement the Porter Stemming Algorithm. This is just for "one" language, and probably does not cover the other "English" dialects. Imagine doing a different workaround for every language in existence...)
RAG is actually a very language-agnostic way to work around those issues.
(Obviously this doesn’t apply to searching actual rich document data - for that, go all in on text search, embedding, etc)
It was one of the most fun projects I've worked on in my career so far. I got a learn a lot about how US and international addresses worked, so many edge cases, and got to really understand how customers were using the existing search to make sure they weren't adding any duplicates to the database. Token filters and synonyms were neat and figuring out the right indexing strategy was a lot of fun.
It was a lot more work to get it right for most of the use-cases our customers had than just "throw it into ES and be done". That would probably have been fine for the 80/20 case, like you said, but I agree that the bulk of the work is going to be fine-tuning the search solution, whatever technology you're using.