KAIST develops CONDA vector search tech that maintains AI accuracy as data changes
In short
KAIST said a team led by professor Kim Min-soo of its Department of Computer Science developed CONDA, a dynamic vector search technology that keeps search accuracy stable when data is continually added and deleted. The team said CONDA improved search accuracy by up to 24.5% and data processing speed by up to 1.90 times over existing state-of-the-art technology, and that it is intended for knowledge search, news search, product recommendation and retrieval-augmented generation.
Read the full story 2 min read
The Korea Advanced Institute of Science and Technology (KAIST) said on Oct. 5 that a research team led by professor Kim Min-soo of its Department of Computer Science developed CONDA, a dynamic vector search technology that keeps search accuracy stable in environments where data is continually added and deleted, Yonhap and AI Times reported. [ 1 , 4 ]
According to the reports, CONDA — short for Connectivity-Aware Dynamic Index — considers not only the distance between data points but also whether search paths are connected well enough to reach the information that is needed. The method keeps important links in place so that particular information is not isolated in the search network when new data arrives or existing data is removed, the reports said. [ 1 , 2 , 3 , 4 ]
AI Times described the problem the technology targets as a “connectivity collapse”: as documents, news and product information are repeatedly added, edited and deleted, the links built between data gradually break, so AI cannot reach information that is still in the database. It compared the situation to a navigation system whose destination stays in place while road information around it keeps changing. The outlet said CONDA rests on three techniques: connectivity-aware search path selection, expansion of connection candidates for new data, and efficient lazy deletion, in which deleted items are marked first and their links are cleaned up gradually during later searches or updates. [ 4 ]
The research team said experiments showed CONDA improved search accuracy by up to 24.5% and data processing speed by up to 1.90 times compared with existing state-of-the-art technology. In a test with 100 million data items over six hours, with search and data updates running at the same time, the team said CONDA kept the shortest response time and high accuracy among the compared technologies, indicating it could be applied to large-scale AI services. [ 1 , 4 ]
The team expects the technology to be used in knowledge search systems for internal corporate rules and work documents, as well as recent news search, product search and recommendation, and as a base technology for retrieval-augmented generation (RAG), in which generative AI looks up external information to use in its answers. Kim said the key value of RAG is that “large language models can immediately find and use the latest information they have not been trained on”, and that the study is significant for presenting a data infrastructure that lets AI use accurate, current knowledge over long periods. [ 1 , 4 ]
AI Times reported that the team also plans to move ahead with applying the technology to services, and identified KAIST president Bae Choong-sik. Chosun Ilbo carried a short summary of the announcement in Korean and Japanese, published at the same time, repeating the claim that the technology maintains search accuracy as data is added and deleted but giving no further figures or comments. [ 2 , 3 , 4 ]
Why it matters
The technology targets a known weakness of retrieval-augmented generation, in which AI answers depend on search finding current information that a model was not trained on. If the reported performance holds, it would matter for enterprise knowledge systems, news and shopping search and other AI services where data changes constantly, though the documents give no independent verification and no timeline for wider deployment.
Key facts
- KAIST announced on Oct. 5 that a research team led by professor Kim Min-soo of its Department of Computer Science developed the dynamic vector search technology CONDA. [ 1 , 2 , 3 , 4 ]
- CONDA considers not only the distance between data points but also whether search paths are connected well enough to reach the needed information, so that information is not isolated in the search network when data is added or deleted. [ 1 , 2 , 3 , 4 ]
- The research team said CONDA improved search accuracy by up to 24.5% and data processing speed by up to 1.90 times compared with existing state-of-the-art technology. [ 1 , 4 ]
- The team ran a six-hour test with 100 million data items in which search and data updates were performed simultaneously, and said CONDA kept the shortest response time and high accuracy among the compared technologies. [ 1 , 4 ]
- The team expects the technology to be used in knowledge search systems for internal corporate rules and work documents, recent news search, product search and recommendation, and as a base technology for retrieval-augmented generation (RAG). [ 1 , 4 ]
- AI Times reported that CONDA was developed as a graph-based vector search technology resting on three techniques: connectivity-aware search path selection, expansion of connection candidates for new data, and efficient lazy deletion. [ 4 ]
Confirmed by several sources
- KAIST announced on Oct. 5 that professor Kim Min-soo's team in its Department of Computer Science developed the dynamic vector search technology CONDA. [ 1 , 2 , 3 , 4 ]
- CONDA takes into account both the distance between data and whether search paths are properly connected so that needed information can be reached, preventing information from being isolated when data is added or deleted. [ 1 , 2 , 3 , 4 ]
- The research team said CONDA raised search accuracy by up to 24.5% and data processing speed by up to 1.90 times compared with existing state-of-the-art technology. [ 1 , 4 ]
- The team said that in a test with 100 million data items over six hours, with search and data updates running simultaneously, CONDA maintained the shortest response time and high search accuracy compared with other technologies. [ 1 , 4 ]
- The team expects the technology to be used in knowledge search systems, recent news search, product search and recommendation, and as a base technology for retrieval-augmented generation (RAG). [ 1 , 4 ]
Still unclear
- Whether CONDA is being applied to commercial services, and on what schedule AI Times reported that the team plans to proceed with applying the technology to services, but no other document mentions deployment plans or timing, so this rests on a single outlet.
- Which technologies CONDA was benchmarked against Documents refer only to “existing state-of-the-art technology” and “compared technologies” without naming them.
- Whether the reported performance figures have been peer-reviewed or independently verified All figures come from the research team's announcement as reported by the outlets; the documents do not describe external verification.
What local media are saying
Timeline, local time
- Yonhap publishes its report on the KAIST announcement, quoting professor Kim Min-soo and the performance figures. [ 1 ]
- Chosun Ilbo publishes Korean- and Japanese-language summaries of the same announcement. [ 2 , 3 ]
- AI Times publishes a detailed article on CONDA, describing its three core techniques and a plan to apply it to services. [ 4 ]