Skip to content

dergigi/ants

Repository files navigation

ants - advanced nostr text search

An opinionated search interface for nostr.

ants logo

ants is the search and discovery interface I always wanted to have. It has all kinds of search modifiers, can do reverse image lookups, and will not shy away from throwing events back at you that it can't even render yet.

The basic philosophy is to always stay in search and to embrace false positives, i.e. rather show too much than too little. But we still want to be able to filter out nonsense and spam. It's very much a work-in-progress. It doesn't have many WoT features yet, for example.

The current version is not very performant and will probably crash often.

But it's useful to at least one person already, which is me.

Search Examples

ants can search for all kinds of stuff by making good use of NIP-05, NIP-50, and having human-readable shorthands for (pun intended) the most common kinds:

Type /examples in the search field to see the full list.

URL Paths

ants supports bech32-encoded entities as per NIP-19, just like njump.me and other portals do:

The /t/ path supports multiple separators (comma, plus, and space).

Relay Logic

There is hardcoded relays for search (NIP-50) as well as for general use.

Upon login, we retrieve the user's relays as per NIP-51 (kind:10002) and remove any blocked relays (kind:10006). We also retrieve the user's search relays (kind:10007) and use them for search queries in addition it to the hardcoded list of search relays.

When connecting to a relay we retrieve the supported_nips as per NIP-11. The relay list as well as the supported NIPs are shown in the relay status indicator. Relays that returned one or more of the results that are currently shown on the page are shown in blue. Relays that support NIP-50 show a magnifying glass. The relay icon in the relay status display allows for relay-based client-side filtering of results.

Search Logic

We have two kinds of queries:

  • Search queries (NIP-50)
  • Direct queries (bech32-encoded entities as per NIP-19, i.e. npub, note, nprofile, nevent, naddr)

Search queries:

  • Connect to NIP-50 relays exclusively
  • Do a NIP-50 search for each resulting query we have
  • (we might need to do multiple queries if the user does an OR search)

Direct queries:

  • Connect to all relays
  • Retrieve the bech32-encoded entity directly
  • (No need for a NIP-50 search)

A lot of complex queries are still direct queries, e.g. is:highlight by:fiatjaf OR #YESTR by:dergigi.com will resolve to two queries, namely:

  1. kind:9802 by:npub180cvv07tjdrrgpa0j7j7tmnyl2yr6yr7l8j4s3evf6u64th6gkwsyjh6w6
  2. #YESTR by:npub1dergggklka99wwrs92yz8wdjs952h2ux2ha2ed598ngwu9w7a6fsh9xzpc

None of these need search. (1) is simply kind:9802 with authors: [6e468422dfb74a5738702a8823b9b28168abab8655faacb6853cd0ee15deee93] and (2) is simply t:yestr with Gigi's npub converted to hex.

However, if we have something like has:video by:HODL we will have to hit NIP-50 relays, because has:video expands to .mp4 OR .webm OR .mov ... and thus we'll have to do a full-text search.

Profile Lookups and Vertex Logic

When resolving a by: or p: search, we try to do a best-effort profile lookup. If the user is logged in we use the Vertex DVM to do the profile lookup, using personalizedPagerank.

In short:

if logged_in:
    profile = get_profile_from_vertex("search string")
else:
    profile = get_profile_from_fallback("search string")

The fallback is a NIP-50 search that attempts to do a "smart" ranking of profile results to figure out the most real (most relevant) profile. But it might be wrong. For reliable results users should login and use Vertex.

Profile searches might be a plaintext search like gigi or dergigi, npubs like npub1dergggklka99wwrs92yz8wdjs952h2ux2ha2ed598ngwu9w7a6fsh9xzpc or NIP-05 identifiers like me@dergigi.com, or top-level NIP-05 identifiers like dergigi.com (which is equivalent to @dergigi.com or _@dergigi.com).

If it's a valid NIP-05 we should be able to get the hex of the npub straight up, without having to hit a search relay. If it's a plaintext search like fiatjaf we basically do a kind:0 fiatjaf, i.e. a NIP-50 search for profile events (hitting NIP-50 relays exclusively).

Live Instances

Ranking behavior

  • When logged in and Vertex credits are available, profile lookups and author resolution use personalizedPagerank (your pubkey is sent as source).
  • When logged out or Vertex is unavailable, relay-based ranking is used (see fallback below).

This applies when resolving usernames like by:john or direct profile lookups like p:john. See the Vertex docs for details on parameters and response format: https://vertexlab.io/docs/services/search-profiles/.

Note that proper username resolution requires Vertex credits. See Vertex pricing for details on credit costs and tiers.

Fallback ranking

If Vertex is unavailable or credits are insufficient (or when logged out), we fall back to a relay search for kind:0 profiles matching the username and rank candidates as follows:

  • Logged in: prioritize profiles that you directly follow; tiebreak by prefix match and name.
  • Not logged in: sort by the number of follower references (count of kind:3 contacts that include the candidate pubkey), then prefix match and name.

Deployment Configuration

Set the public site URL (used for Open Graph/Twitter metadata) via environment variable:

NEXT_PUBLIC_SITE_URL=https://ants.sh

You can place this in a local .env file.

Search substitutions

All search substitutions (site aliases, media type expansions, etc.) are loaded from replacements.txt. This file contains the mappings for site:, is:, and has: modifiers, making it easy to see what substitutions are currently available and add new ones.

Here are some excerpts:

...
site:gh => (github.com OR www.github.com OR gist.github.com)
site:quora => (quora.com OR www.quora.com OR m.quora.com)
site:hackernews => (news.ycombinator.com OR www.news.ycombinator.com)
site:hn => (news.ycombinator.com OR www.news.ycombinator.com)
...
has:video => (.mp4 OR .webm OR .ogg OR .ogv OR .mov OR .m4v)
...
is:profile => kind:0
is:tweet => kind:1
is:repost => kind:6
...
is:highlight => kind:9802
is:blogpost => kind:30023
is:muted => kind:10000
...
nip:99 => nips/blob/master/99.md
nip:B0 => nips/blob/master/B0.md
nip:C0 => nips/blob/master/C0.md
nip:EE => nips/blob/master/EE.md
...

It's probably very stupid to do it this way, but I went with the flow and stuck with it. In the future each line might be a nostr event.

TODOs, aka what I wanna do next

  • add support for code snippets (kind:1337)
  • don't do so many requests, lots of requests can be merged into one
  • add a /kinds command that shows all substitutions
  • implement streaming search aka "live" mode
  • support relatr in addition to vertex
  • be nice to relays (respect limits etc)
  • add proper support for blog posts (kind:30023)
  • add "blossom search" to images (via sha256 hash)
  • explain what the different icons and symbols mean somehow
  • move some things around in the UI
  • make stuff less stupid and buggy overall

References

License

MIT License. See LICENSE for details.


original ant and loupe vectors by joko sutrisno and noodledoodle

About

ants: advanced notes and text search

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages