Hacker News new | past | comments | ask | show | jobs | submit login
Launch HN: Needl (YC S22) – Simple search across all your apps
106 points by MaxKeenan on Jan 31, 2023 | hide | past | favorite | 75 comments
Hi HN, we’re Max and James, co-founders of Needl (https://www.needl.tech). Needl is a single search bar for all your apps that lets you find any document, message, or file you need. Here’s a demo video: https://www.loom.com/share/652ae56c8ee34bc2a84b8aacf479a9f4.

We’ve all struggled to find information and couldn’t remember if it was shared over Gmail or Slack or Notion. Most of us have hundreds of thousands of emails, documents, messages, etc., and only a small fraction are actually useful. Internal search is even harder than web search in some ways: with web search, there are dozens of right answers and millions of previous searches to train ranking models, but internally (i.e. on your own data), there is only one right answer and limited training data.

Needl’s value prop is twofold: it unifies all of your information together and provides better search than existing platforms (here’s an example of Needl compared to Gmail: https://imgur.com/a/3ZX8f8Q). Together, these save time by giving you exactly what you need. Recently we also launched a personal assistant that gives direct answers to any search. Simply ask, “What’s the budget for our 2023 team offsite?” or “When’s my next flight to SF and what’s the confirmation #?” From your web apps, we’ll generate the exact answer you need. (pro tip, you don’t even have to phrase it as a question).

To rank results on Needl, we combine text matching, semantic search, and platform-specific usage patterns (e.g. email opens). Semantic search (based on user intent / meaning) has helped us solve the hard problem of internal search – everyone has a different vernacular and way of organization. We use LLMs to create embeddings and then use vector search to allow greater breadth of results over traditional text-based search. Our search (and generative answers) even works well in different languages, something that surprised us.

To give generated answers to questions, we take the first results on Needl and run them through GPT-3 and use an engineered version of the user’s query to prompt GPT-3 accordingly. It’s similar to what Phind.com (Hello Cognition) and Perplexity.ai do with web search.

Using LLMs to build relationships between documents for search and recommendation was a remnant of our failed college startup. We had built a “TikTok for blog posts” that recommended user-written content based on their interactions with other posts. We quickly found out social media moderation was a nightmare (who would’ve thought) and few people enjoy reading any more.

It wasn’t until we threw in the white flag and went to our full-time jobs — James as a SWE at Microsoft and me in investment banking at Moelis — that we realized that recommendation + search was the real product. At my job in particular, intranet search was unusable. The primary way people “searched” for information was by sending company-wide e-mail blasts asking if anyone had materials about an industry or company! This got us wondering why internal search sucked so badly and led us down the path of building Needl.

Obviously with a product like this, privacy is the #1 concern for users. Some companies have solved this concern by going local first, but that involves significant reduction in quality of search, because LLMs can’t be run locally. We went the opposite route: users’ entire index is stored on the cloud. We know that’s not for everyone, but the results it enables are significantly more powerful, and we make clear commitments to our users: (1) we never access users’ personal information without explicit permission; (2) all information is encrypted both in transit and at rest, and (3) we will never sell users’ information. We’ve gotten SOC 2 Type II compliant to give our users assurances about the way their data is being handled and our dedication to information security.

Setting up Needl takes less than 3 minutes—users create an account, grant read access to the platforms they want to search through (Slack, Notion, GSuite, etc.), and we index their information. Once completed, you have instant, cross-platform search in a Spotlight-esque format—control + space opens Needl from anywhere.

In terms of pricing, Needl is free for up to 3 integrations and $10/month for unlimited integrations.

To learn more, you can download and set up Needl at https://www.needl.tech/. If you’ve already tried tools in this space, we’d love to hear your experience and what you care about most. We look forward to everyone’s comments!




The core idea is similar to usefyi which pivoted to nira, an access/authorization monitoring service. Here is a thread on why they made the pivot.

https://twitter.com/hnshah/status/1399773415213436934

How is needle different?

Are you also competing with various ctrl + K spotlight competitors, many funded by YC like raycast?


Thanks for sharing, definitely familiar with Usefyi -- them and Daniel Gross' Cue really set the stage for this space. In our mind, the biggest limitation for internal search is search quality, something that is becoming more and more improvable with LLM development


Immediate intention is to not compete with Raycast and the like. Search itself is a too difficult problem to also go down the path of fully replacing Spotlight.


Thanks for the answers. I wish you have a successful launch.


Thank you!


I always found this class of product fascinating, as new ones pop up all the time. I remember...

* Google Desktop

* Grepline / Cue (this is W10)

* ... various other products ...

I feel like for personal users, the product ultimately isn't sticky enough.

However, "enterprise search" feel more viable path especially this is where it becomes more valuable (and where the money is). Also anybody whose working at a large company would need to have some enterprise procurements to sign up for such a service anyway. I'm talking about players like Glean in this area.


Good point. It's interesting because, in many ways, search is easier at the enterprise level than the individual level. The usage patterns & social graphs of large orgs is incredibly helpful for relevance ranking. For example, if someone on your team just accessed a doc, the odds you are looking for that doc are pretty high.

Our thesis is that Google Desktop + Cue and the dozens of others lacked was good enough search to be sticky. We're not entirely certain this is the only reason, but we're testing it out.


Congrats on your launch, looks like a very useful tool!

Although I have to say the privacy implications are quite concerning before I'd consider using it personally or for a team.

You mention at rest encryption, how does that work with indexing the data and making search work?

Will you offer retention policies on data?

Any plans to include direct messaging platforms (if they offer api's)?


Understand where you're coming from a privacy perspective. Our encryption operates at the file system/storage device layer level so it doesn't affect our index at all other than a small I/O performance hit when storing and accessing data.

W.r.t. retention policies, information is immediately deleted upon account deletion. We also delete all data after account inactivity for 2 months.

Definitely exploring messaging platforms, interestingly enough iMessage (despite not being available via API) can be indexed locally on Macs. Just not something that's on the top of our backlog. Thanks for your thoughts!


> Our encryption operates at the file system/storage device layer level so it doesn't affect our index at all

This means you store most data in plaintext in the index? Enough to reconstruct most of the content?


Yes. We are SOC II certified and have strict limitations on who can access that data and under what circumstances (only if given explicit permission by the user). Definitely understand that could be a deal breaker for some, but with current methods, we believe that this approach will allow us to deliver more powerful search and greater value in the long run.


I really love the product. Would you be open to hiring an intern to work on the technical aspect but also dabble with the product? Im currently a 3rd year CS + Philosophy student in NYC. If its something you'd consider can I have your email and ill reach out with some more info about myself.


They have an email address on their website.


Needl looks really cool, congrats on the launch!

Wondering if you could speak on a few topics in the spirit of hacker news / yc community:

1. Wondering if you've run into any friction yet where employers block the end-user from integrating with their apps? Just curious to what level that is or is not a common practice for enterprise users?

2. After using previous products that promise similar "search across all apps" functionality I soon felt the early excitement/novelty wear off and returned to old behaviors (ie, searching using each app's search). No doubt this is just a testament to how hard new habits are to build (and the old to let go of) but I'm wondering what your thoughts are on the human behavior / psychological aspect of products that have characteristics like this? As a tangential example, recent blogs posts featured on HN talked about how perhaps a chat-gpt conversational interface is not actually ideal to replace various UI/UX as it may take more cognitive overhead for the user to think of how to describe what they want when normally the UI/UX is what previously assisted them to arrive at that data. Basically I'm wondering how congruent (to existing behaviors) or novel your UX is and to what extent you believe this presents challenges to your team when designing (and perhaps marketing) the product?

3. Can you speak on your YC journey?

Thanks and good luck!


To the first point -- certainly some friction here. In most cases the employee is able to integrate the app and flags arise to IT after the fact. Then the employee connects us with IT and we resolve the issue by going through a security questionnaire / showing our SOC II. The majority of our current users are SMBs, where they don't have significant IT oversight, we're one of the few products in the space that's truly self-serve.

To the second -- very good point and something we continue to experiment with. Our basic thesis is -- if your search is really good, people will learn to default to your app. For many of our users, the unification is less so the value and more so the better search quality, especially compared to Gmail and GDrive. Another good point that writing things as a question is often a worse user experience than just using Google search esque terminology. For our Ask Needl feature, you can still get value without writing an entire question. Ex: website total budget --> $1000-1500.

Our YC journey was certainly formative. Much of the advice given was counter-intuitive in the bull market of Nov 21 when everyone was spending and growing as fast as possible. My biggest takeaway was the focus on solving hard problems with small teams and only caring about what your users think of you. Would highly recommend - the culture is unparalleled.


Love the concept. I wish there were a way to add custom integrations for in-house software we use or uncommon systems we use. It would work better if these things were configured at the organization-level and not user-level (i.e. IT connecting Confluence to Needl instead of individual users requesting access one-by-one).

Typo on the pricing page: "and soon to be may others", I assume meant "many".

Would love to try but my daily driver is a Linux machine. I just don't spend enough hours a week on my Macbook for it to be relevant right now. Edit: I'll try the Chrome extension.

Edit: The hover for "Free for 14 days" in the pricing page is black-on-black unless you hover exactly on the text. Webflow hover styling is awful, happens to me often too :)


Understand where you're coming from on the org-level auth vs individual. Part of the goal with Needl is to be entirely self-serve, where one member of a team can use it without needing company wide approval.

Thanks for calling out the site issues -- will get those fixed asap!


If one member of a team is meant to be able to just use it without IT being involved, don't you foresee companies getting very cranky about an employee uploading all their work documents, emails, etc. to an external cloud they don't control?


Certainly happens - in this case IT usually contacts the employee, who connects us with IT and then we share our SOC II / describe our security processes and they grant us company wide access. Ends up not being a massive problem, but the added benefit on an employee / individual being able to self-serve offsets it, in our mind at least.

At some point, we'll likely kick off an enterprise-focused initiative. Thanks for bringing this up, very good points.


I just emailed you for help. When trying to self-serve, I immediately got blocked by automated corporate policies :)


Yup! Hopefully the workaround I sent was easy enough -- please let me know if there's anything else I can help with!


Is there a way to switch between "All", "Files", "Messages", etc. using the keyboard? I'd like to be able to quickly filter the search results when I know what kind of thing I'm searching for.


Really good point -- will definitely add in the next few days. What hotkey are you thinking of? My immediate first thought is tab -- think that would work well for you?


Yes, tab is what I instinctively reached for. That seems to be the intuition.


Just wanted to close the loop here -- we just added this in our latest push today. Thanks for the suggestion!


Good luck with this, looks good, and an interesting project from a tech creator perspective. Particularly like the reassuring notes about auditing etc being central.

Minor note - the link to try the web version doesn't work for me from the pricing section free option down the page. Chrome/Firefox Android mobile and desktop view. The main link at the top works fine.


Great catch -- will definitely fix. Thanks a lot


Congrats on the launch. I'm not sure if I'm the only one, but I never had the need or desire for this type of product. I usually know if the thing I'm looking for was in an email, in my Basecamp or in a Google Doc. I wonder why I feel so differently than obviously many other commenters on here.


I don't think this is controversial at all. Many of our users are the exact same way. For them, the value add can be a few different things: (1) searching across multiple accounts on the same platform, (2) better search than existing platforms (here’s an example of Needl compared to Gmail: https://imgur.com/a/3ZX8f8Q), and (3) general speed improvements.


Looks good, will definitely give it a try. I'm starting to have so many apps that I trigger with keyboard shortcuts (Raycast, etc)...getting more difficult to choose shortcuts that don't conflict and remember all of them.


Fair point. Some users set Needl as their default tab via our chrome extension -- https://chrome.google.com/webstore/detail/needl/hepjidniooep.... Can even open quick search via the chrome extension in the browser too.

Others just use the web app -- https://app.needl.tech/


Yeah, actually I was surprised that ctrl + space didn't conflict with anything I already have, unless I'm in Firefox, in which case it conflicts with the "List all tabs" search thing.


This tool is looking really neat but I'm on Linux using Firefox and Android using Firefox so no luck. Web app is also working bad in Firefox on Android.


As an aside, is there a selfhostable FOSS alternative for this product?


I really like this idea! I definitely think this product has some potential. A few quick questions:

1. Does this integrate with the filesystem (specifically with MacOS) or iCloud Drive? 2. Are there any future plans to integrate with iMessage? While perhaps less important to enterprises, I feel like this could be very useful for “prosumer” productivity tool users like myself. 3. Are there any future plans for a mobile app (specifically on iPhone).

I’m excited to try this product out. Congrats on your launch!!


Yes! We integrate with Local, though iCloud is much more locked down. We can index local files that are then offloaded to iCloud, but can't directly index iCloud.

iMessage is something we've explored, also locked down like iCloud with no API but there are some workarounds by using what's locally stored. No plans for a mobile app, though or web app works surprisingly well on mobile https://app.needl.tech/.

If more stuff comes up as you're using Needl, would love more of your thoughts and feedback. Can always reach me at max@needl.tech


Feature request: Index arbitrary content/history across browsers and allow it to be re-opened (my main use case for your pseudo-competitor Rewind.ai).


Thanks for sharing -- makes a lot of sense. Indexing browser history is very high on our priority list, lets us cover the less-widely used apps much easier. (though is somewhat limited to things you've seen while the integration is set up, similarly to Rewind)


Congrats! I also think that your name (needl) is very creative and fits the problem you are solving very well :)


With companies getting hacked left and right, I'm curious to hear how you manage to break through privacy/security concerns that I imagine would pop-up in a prospect's mind when they realize you're doing everything on the cloud.


What does the prompt injection look like, how do you prevent hallucination


We've experimented with this a fair bit, very low occurrence of the model making up facts when the answer exists within a document, but users have reported outside information (general knowledge) being shown in their results.

We essentially just prompt GPT-3 to ignore everything that is outside of the chunks of information we provide it.


Are there plans to back up the "suggested answer", which I presume is LLM generated, by a definitive source? The first question in the demo returned the relevant document you were looking for, but I didn't see this in the search results for the second question.

I'm not sure I would trust a system like this unless I could click through and see the source of the answer I'm reading, and make sure that the LLM is referencing the correct email/document.

This seems to be a common growing pain in places where an AI model is expected to provide authoritative answers - I wonder if (at least in your case) it's possible to use a more traditional fuzzy search algorithm to attempt to find the source, based on the LLM's answer string.


Currently something we're working on with prompt engineering, but love your suggested approach. We'll definitely look into that more -- thanks for sharing.

For now, the search is always generated from the first 5 or so results. So you always have an idea of where it's coming from.


How is this different than Dropbox Dash (formerly known as Command-E)? What I love about Dash is that no metadata/index leaves my computer - are you taking the same approach?


We went the opposite route: users’ entire index is stored on the cloud. We know that’s not for everyone, but the search results it enables are significantly more powerful, and we make clear commitments to our users: (1) we never access users’ personal information without explicit permission; (2) all information is encrypted both in transit and at rest, and (3) we will never sell users’ information. We’ve gotten SOC 2 Type II compliant to give our users assurances about the way their data is being handled and our dedication to information security.


Got it. FWIW, the search experience with command-e has been fantastic and super powerful since the beta, so I'm not sure how much more powerful this is to require me to leak all my info to the cloud. It'd be hard to believe it's a 10x improvement e.g.


Cool to hear Command E search has improved a lot -- it's good for the space generally. Curious what platforms you're searching mostly on -- for the majority of Needl users their information is already on the cloud (granted a startup having your data is different than Google).

We've also found searching messaging platforms (Gmail / Slack) and the generative aspect to be the most difficult. Finding links to docs that you know the title of is less challenging. So if you just use Command E for Dropbox / Drive / Local, then it's almost certainly the right solution for you.


How do you compre to https://slapdash.com?


Biggest difference is search quality. Our entire product is focused on making information more accessible across web apps. We have generative Q&A search and semantic search, something Slapdash does not offer (to my understanding).

On the other side, we also don't offer the Command Bar-esque features that Slapdash and Raycast offer. For now, our #1 focus is search and search alone.


Thanks for this, this one has a Linux version, already better in my books.


I still miss my Google Desktop!


Interesting but, no Linux version? Or does it run under wine?


Hey Max! Great seeing you launch here, congrats :)


Looks great


Good luck on your launch!


If you haven't seen them already, the Danish startup Raffle.ai are solving a similar problem of internal search


Welcome to the space - I'm one of the founders from https://curiosity.ai! It's crazy how much movement there is recently, with Slapdash (now part of ClickUp), Command-E (sold to Dropbox), Raycast and many others, seems like the quest for the one search is still pretty much alive!

Really interesting to see how you're integrating LLMs - something we've been curious to play with (but not yet feasible on-device as our app indexes everything locally).


Certainly a very lively space, feels like just about everyone is trying to solve the problem at hand. Always happy to share thoughts / learnings in the space -- we should find a time to connect, feel free to email me at max@needl.tech


Awesome, dropped you an email - let's find some time to catch up over some virtual beers!


Congrats on the launch Max and James!

We've been in this space for a lil bit too with eesel (https://eesel.app) and it's really cool to see more people tackling these problems. It's about time we fix this.

Keen to be inspired and learn! :)


How does your product compare to https://haystack.it?


Not familiar enough with them to say -- doesn't seem to be publicly available.


Here at Haystack, we provide an industry leading cross-platform application to help obfuscate, disguise, and confuse your data that increases security by decreasing its ability to be found in a search.


Hi I'm Yuval, one of the devs behind haystack, I messed around with needl + other alternatives (e.g unleash, getcommande, glean).

I talked to Max as a user over needls privacy policy.

The biggest difference is that haystack gives you search results relevant to your query. Just like google.

needl and alts are more of a spotlight/raycast alternative, great as a productivity tool, they require you to know exactly what you're looking for beforehand, in my opinion which leaves more to be desired as a search product.

Another big differencee is around data privacy, haystack is open, free and most importantly self-hosted.

Also we allow building low code custom data connectors into haystack!

It'll become publicly available next month here:

https://github.com/haystackoss/haystack

Thanks :)


[flagged]


Having just looked at the data, it's pretty clear to me that this entire exchange was fake. We ban accounts that do that. Please don't do it again.

It's particularly uncool that you're doing this, given the number of times that you posted at length accusing us (falsely) of nefarious actions in moderating this site.

Btw, I took special care to make sure that your own startup got a nice amount of exposure on HN, despite the fact that the posts were obviously ringvoted. We make a point of not holding grudges and try to make sure that good, interesting content gets seen. But please stop using manipulative promotional tactics either for your own stuff or your friends.


So he made a fake account to ask the question then replied with his original account?


No, I meant that (assuming I'm reading the data correctly) the exchange was orchestrated and not at all organic.

I don't normally post publicly about these things but when people try to mess with another startup's launch thread, it gets my goat (YC-related or not), and to use manipulative tactics while doing it goes over my threshold for saying something.


I'm not sure of the data, I was linked this by a friend, I'm not really sure how a proprietary saas company ends up as the good guy, while we're trying to ensure the next big search platform remains open, end up as thugs.

It seems the main mods problem with what I did is plugging our github url, which made it obvious I diverted attention from needl's launch, mistake I made. I'm not deleting any comments, because I prefer context to mod interactions stand in broad daylight.

I'm truely sorry, I like this forum, mods here are awesome.


Don't worry! as long it doesn't happen again, it doesn't need to be a problem.

Obviously I can't know what happened off the site, but to judge by what I'm looking at in the data at our end, this exchange was obviously not organic, nor was this the first evidence of collusion that I found.


[flagged]


Please stop hijacking another startup's launch thread. You've already posted elsewhere in the thread, and that part was pretty obviously a fake, staged conversation. Not cool. (In case anyone's wondering, yes, this is just as not-cool whether the startup is YC funded or not, and I've posted such scoldings in such threads in the past.)

Manipulative tactics like voting rings and staged comments are not ok on HN—this is in both the site guidelines (https://news.ycombinator.com/newsguidelines.html) and the FAQ (https://news.ycombinator.com/newsfaq.html) so please don't do those things again.

You already had a nice and prominent Show HN for your own startup just a few weeks ago—one which, by the way, we helped out by putting it in the second chance pool (https://news.ycombinator.com/item?id=26998308).

(We detached this subthread from https://news.ycombinator.com/item?id=34599406.)


Was not my intention at all, sunlight is the best disinfectant.

I'm not saying you're accusing me of anything, but what would be a fair comment in you're eyes, I'll be happy to learn.


What would be fairest is to let a competing startup have their launch thread without trying to interfere with it. Each startup deserves its own day in the sun.

If you're going to post, you certainly shouldn't do it more than once.

Replying in multiple places to change the subject to your own promotion is in bad taste, and orchestrating exchanges with seemingly-unrelated other users is an outright abuse and the sort of thing we ban accounts for if they do it repeatedly.


Just to make it clear: I'm not involved in Haystack. I commented here and shared a link on their discord, so someone from their team could have an opportunity to answer

By the way, thank for helping our own startup, we are really grateful for all the attention and encouragement we have received from HN community

My apologies to you, Max and James. This is not how I expected things to turn out


My bad, thanks for letting me know.

I tend to see Needl as peers, not competition, sometimes peers behave in bad taste.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: