New Feature: Simplified Search Experience

[leobalter]

Hi npm users,

We're excited to share that we're piloting we've just released a new and simplified search experience on npmjs.com! 🚀

• changelog
• documentation

This update introduces objective sorting options designed to help you discover the right packages more efficiently.

What's New?

• Objective sorting options:
  • Relevance (based on word match)
  • Downloads (weekly and monthly)
  • Most dependents
  • Last published date
• Streamlined experience:
  • Simplified sorting by removing less specific criteria like popularity, quality, and maintenance.

This feature is currently being tested with a select group of GitHub Stars and high-impact npm users. We're also progressively rolling it out to a subset of users, so you may already see the new search experience if you're logged in.

We're actively collecting feedback during this private preview, so you might notice refinements as we make improvements.

How to Share Feedback

We'd love to hear your thoughts! Please share your feedback in this thread. Your input will help shape the future of the npm search experience.

Thank you for being part of the npm community and helping us improve! 🙌

[qgustavor]

I checked the new experience and missed the "popularity, quality, and maintenance" thing. I can understand why it was removed (not being clear for some people) but for me it was "popularity = score better those with more downloads", "maintenance = score better those which were recently updated" and "quality = ???" (ok, this one was not clear at all, maybe it scored better those with more dependents?).

The current filters are more clear, sure, but I don't think it made looking for a package easier. Example: if I want a modern library (e.g. I want something using ES modules) I could look for the most recent updated, but since it's a strict filter (it sorts by updated date and nothing else) it returns anything which is new, while I would prefer something popular (so well tested by other people) and new. If I only check the most downloaded filters it will return a lot of well-tested libraries, but they might be 5 years old libraries using CommonJS and proto.

The sort by dependents option, IMHO, is only useful for people looking for libraries to use on libraries as it looks like npm only counts library dependencies (not private projects). Example: npm only lists 29 dependents for megajs while GitHub lists more than 8000 (and surely it is not counting a lot of projects which were not published on GitHub).

What I suggest as improvements for the above issues: people could comment here what objective and easily indexable things they look for in packages and allow those things to be set as filters. My suggestions: kinds of module systems supported, package sizes, date of last publish. An example: if I want to find a package for a Vite compiled web app, I can select those with ES modules for almost-guaranteed tree shaking, I can choose <10 MiB package size to avoid bloating the web app, and updated in the last three years to avoid something that uses jQuery or Moment.js, then sort by relevancy because the previous things are filters, but I still want to find my query.

I tested the current system looking for "mega" (as megajs is the most popular package I maintain): on default sort it puts my package in second place, placing the original (and not maintained) project "mega" first, sorting by most downloaded puts it on 9th position, despite the first 8 libraries on the search don't containing the word "mega" anywhere visible on the shown snippet (thus looks like it ignores search relevancy completely), same applies when sorting by most dependents and recently published. About the "mega" project being first, that's fine... I guess: it says it's an exact match and tonistiigi never marked his library as deprecated or archived, he only added a "THIS REPO IS NOT MAINTAINED." to the GitHub repo. But the other libraries could include something to show why those were selected despite not having "mega" shown anywhere in the result. Also, this shows that only adding "objective sorting options" isn't ideal if those sorting options ignore the search query or don't show clearly if they are following it.

UX-wise the new system looks worse for me too as sorting requires two clicks instead of one and I see a lot of empty wasted space in the new search. The sidebar could be restored and the filters placed there, to avoid the extra click to open a dropdown. On small screens the sidebar could be minimized to a dropdown, but, anyway, isn't like the current search is well tested on small screens, the way the element that holds the "username, version, creation/update time" wraps on small screens isn't ideal.

Well, do you know something can be a good inspiration? GitHub Forks page! Look those neat filters!

[leobalter]

Thanks @qgustavor! All the suggestions are great!

My suggestions: kinds of module systems supported, package sizes, date of last publish

Yes, we have just a few other filters like keyword: and maintainer: - I might need to publicly document those - but I'm internally thinking ways to improve how we see and what we should know about the packages. Filters could definitely play a nice part here.

On the other hand, maintenance (or the lack of) is a bit more complex. Some packages are self resolved / evergreen and might not need changes for years, so time without a new version does not necessarily reflect missing maintenance.

[sinansonmez]

one feedback here: https://github.com/orgs/community/discussions/146157

[cressie176]

Hello,

I'm sorry to say but I'm not a fan of the the new Search experience. I'm the author of the RabbtiMQ client rascal and since the changes it no longer appears in default search results for "rabbitmq" despite this being one of the tags, Previously rascal was in the top 5 with 70K downloads a month. When changing the order to "most downloaded this month", rascal appears below libraries like vasync and slugid despite them having no connection to RabbitMQ at all.

I also maintain a relational database migration library called marv that received around 8K downloads a month, and was in the top 10 if you search for "postgresql migration". Now with even more specific keywords it's relegated to 62nd, and below completely irrelevant search results like line-replace, @tomei/sso and timezone-support.

[leobalter]

I understand the experience may not seem ideal for all the cases, but it's a cost for balancing out fairness what it goes for word searching.

I took a quick look on the rascal package and the rabbitmq keyword is not much frequent or highly featured in the description or readme. Analogue to this, I get similar results when I search for rabbitmq site:npmjs.com on Google.

The Default sorting option will bring in results based on the keyword search, and you can find your package while sorting by download counts.

[expert2020h]

What reasoning does the maintainer give for the current search behavior?

[cressie176]

Hi @leobalter

I understand the experience may not seem ideal for all the cases, but it's a cost for balancing out fairness what it goes for word searching.

This doesn't explain why searching for "RabbitMQ" returns libraries like vasync and slugid which have absolutely nothing to do with RabbitMQ, nor reference it in their source code, documentation or metadata. NPM search is clearly broken.

[leobalter]

Unrelated packages are definitely not something I want to see in the results if there isn't any context connecting them, that doesn't mean other specific packages should rank better. It seems the issues are orthogonal.

I see you updated your package and it is showing on the first page when searching for "RabbitMQ".

[cressie176]

Hi @leobalter,

I was just using rascal as an example to demonstrate that since the changes, desirable search results are appearing below far less desirable ones. The problem is perhaps better demonstrated if you search for "hierarchical configuration".

Default

rank	library	comment	relevant	current	popular
1	node-env-configuration	A hierarchical configuration library with 4K downloads, last published 8 years ago	✅	❌	❌
2	ngx-access	An access control library for Angular with 78 downloads, last published 4 years ago	❌	❌	❌
3	turing-config	A hierarchical configuration library with 30 downloads, last published 7 years ago	✅	❌	❌
4	typeconf	A hierarchical configuration library with 21 downloads, last published 7 years ago	✅	❌	❌
5	config	A hierarchical configuration library with 6M downloads, last published 5 months ago	✅	✅	✅
6	d3-hierarchy	A library containing layout algorithms for hierarchical data with 17.6M downloads, last published 3 years ago	❌	❌	✅
7	config-core	A hierarchical configuration library with 20 downloads, last published 4 years ago	✅	❌	❌
8	@ehosick/config-core	A republish / duplicate of (7)	✅	❌	❌
9	nconf	A hierarchical configuration library with 3.5M downloads, last published 1 year ago	✅	✅	✅
10	fconf	A hierarchical configuration library with 11 downloads, last published 5 years ago	✅	❌	❌

Only 8 of the above results are relevant, 6 have fewer than 100 downloads and 7 haven't been published within 3 years. Only 2 arguably justify a top 10 spot and more relevant, current and popular libraries like cosmiconfig (61M) and dotenv (42M) do not even feature on the first page.

Most Downloaded This Week

rank	library	comment	relevant	current	popular
1	commander	A cli library with 165M downloads, last published 7 months ago	❌	✅	✅
2	execa	A process execution library with 87M downloads, last published 1 month ago	❌	✅	✅
3	schema-utils	A webpack validation library with 80M downloads, last published 1 year ago	❌	✅	✅
4	strip-json-comments	A library to remove comments from JSON files with 62M downloads, last published 1 year ago	❌	✅	✅
5	cosmiconfig	A hierarchical configuration library with 61M downloads, last published 1 year ago	✅	✅	✅
6	eslint	A linting library with 45M downloads, last published 1 week ago	❌	✅	✅
7	dotenv	A hierarchical configuration library with 32M downloads, last published 3 days ago	✅	✅	✅
8	diff-sequences	A library for comparing sequences with 41M downloads, last published 1 year ago	❌	✅	✅
9	@eslint/eslintrc	A linting library with 36M downloads, last published 1 month ago	❌	✅	✅
10	css-select	A CSS selector compiler with 32M downloads, last published 3 years ago	❌	❌	✅

Only 2 of the above results are relevant.

Conclusion

The new npm search algorithm returns either

a) largely relevant, but unpopular and unmaintained libraries, or
b) largely irrelevant, but popular and well maintained libraries

Neither option is useful

[cressie176]

For example...

The jsonwebtoken library has absolutely nothing to do with PostgreSQL, or even databases. The README contains a prominent title "Migration notes" which could be throwing your algorithm off.

[qgustavor]

I worked with implementing search on websites before — in fact I changed search libraries around three times just on a single project trying to solve issues related to it — and I know that's hard to implement. That being said, the lack of a search snippet showing the query term emphasized makes many of the results seem completely unrelated. A search snippet, I know, can be hard to implement, the library I ended up using don't support it and I had to implement it myself. There are other things to consider too, like weighting the package description more, ignoring parts of the readme (using a regex or... well... AI).