Looking for Hidden Gems in Scientific Literature

>There are on the order of 100 million papers [reference 2] published to date.

Does anyone else feel as if this (admittedly rough) estimate is off by an order of magnitude?

Note that Claude Shannon's MS thesis was about re-discovering the work of an obscure British analytic philosopher, whose work from about 100 years earlier had been almost completely forgotten. (perhaps a few philosophers and mathematicians remembered Boole, but they certainly didn't teach his work to the engineers who had to design relay-based logic circuits back in the pre-transistor days)

Excellent Article! Definitely needs to be read a few times to get the gist.

In this context folks might find a previous methodology from the Soviet era named TRIZ highly relevant - https://en.wikipedia.org/wiki/TRIZ

TRIZ (/trɪz/; Russian: теория решения изобретательских задач, romanized: teoriya resheniya izobretatelskikh zadach, lit. 'theory of inventive problem solving') is a methodology which combines an organized, systematic method of problem-solving with analysis and forecasting techniques derived from the study of patterns of invention in global patent literature.

TRIZ developed from a foundation of research into hundreds of thousands of inventions in many fields to produce an approach which defines patterns in inventive solutions and the characteristics of the problems which these inventions have overcome.

References:

TRIZ 40 Principles examples for various Domains - https://web.archive.org/web/20111203105442/http://www.triz-j...

TRIZ and Software - 40 Principle Analogies, Part 1 - https://web.archive.org/web/20120130205515/http://www.triz-j...

TRIZ and Software - 40 Principle Analogies, Part 2 - https://web.archive.org/web/20120131003258/http://www.triz-j...

Been there done that. At least for life science / health publications. The article is spot on.

Not sure if there is value of that approach in other more rigorous fields but in health for sure it does. The knowledge in health science is generally fragmented and a way to connect islands of knowledge has the potential to unlock a lot of value.

If you would like to see how this article ideas are applied in a playful manner in a web application you can visit: https://www.biovista.com/vizit/

For those interested in delving further into LLM Scientific Discovery there is a great github repo grouping research papers on this very topic - https://github.com/HKUST-KnowComp/Awesome-LLM-Scientific-Dis...

Personally I'm a proponent of representing academic knowledge in knowledge graphs, and this site does just that - https://orkg.org/

I've just launched a site to find code repositories linked to academic papers and to summarise key paper attributes. In the future I intend to integrate a hypothesis generator - https://researchlit.com