How do you even compare search engines? For the longest time, this question was out of the question (sorry for the pun ?) because Google dominated this category so hard it wasn’t even worth asking.
Since the LLM boom, when it was clear that asking ChatGPT questions in natural language is much preferable to contorting oneself to create the best search query, folks started seeing the potential end to Google’s reign of the place that answers your questions.
Heck, even Google in the last Google IO said they are working on a new feature that will “let Google do the Googling for you”
I’ve been a long time user of Perplexity Pro and honestly I love it, I’ve set it up as my default search and use it quite often, and the latest research mode update was really helpful for me to research a family vacation to Dubrovnik, discover day trip requirements and much more!
So I was quite surprised and excited to see that OpenAI announced SearchGPT, their temporary prototype of a reimagined AI based search and QUICKLY signed up to the waitlist, because, well, I’m into testing AI things, so here we go, here’s a quick comparison between the two AI powered search engines that you all asked for ?
* This is to acknowledge that Perplexity is a mature product that has an amazing team and SearchGPT is a temporary prototype from a company that does TONS of other stuff
Review approach
While the video above was recorded in 1 shot and you can see me running through a few examples, in the post below I will try to review and compare both search engines on the following criteria (you can consider this the table of contents for this post)
UI & UX
Real Time Search Results
Local results
Complex & Multistep queries
Multilinguality
Widgets & Share-ability
(there are definitely more ways to compare and evaluate, please feel free to leave comments as to what else I should test with SearchGPT)
UI & User Experience
The first thing I Immediately noticed was the super sleek UI, there are barely any elements on the page, you type and you get an answer. There’s also a typeahead suggestion as you type, which Perplexity doesn’t have (yet, they apparently just launched it in beta to some customers)
On the downside, there’s no search history tabs, no settings, no knobs to tweak.
The main User Experience wow effect comes from the speed, SearchGPT is FAST, it feels faster than Perplexity, and while it’s not faster than Google, Google just gives me links (at least for the queries I tried, did they completely remove their UI after the embarrassing responses with the pizza glue incident?) and SearchGPT answers complex queries super quick which matters a LOT for user experience.
SearchGPT also has a header that summarizes the answer in one sentence, which is very helpful to quickly glance at. On the left sidebar there are a few buttons for a list of sources for the current search and images. And that’s basically eat.
I’ll give this category to Perplexity for now, just because it’s a fleshed out product, has a mobile app, has follow up questions, and just feels more mature, but I really like the simplicity of SearchGPT!
Searching for stuff that JUST happened
The first thing I wanted to test is how up to date SearchGPT is compared to Perplexity, given that OpenAI just launched it (but has a Bing backing) it’s interesting to see how fast they are indexing things that just happened
What is the price of Bitcoin?
This one was difficult to even test, because well, both got it wrong, while both claimed they had live data (which none really did, CoinGecko showed a different price) and bitcoin fluctuates quite a LOT.
Does Alex Volkov host a course for Weights & Biases?
Friday, my teammate Agata surprised me by saying she used a bunch of my videos and stitched them together into a course about our product Weave (which you can see here btw). Since I know that this page went live yesterday, I wanted to see how fast both these search engines index that new page.
Both fail to find it, but what’s really interesting is, during the recording of the video above (which took place 5 hours before I’m writing this), Perplexity DID have the right answer and did find the right course with the right citation (so did google) so I don’t know why did Perplexity decide to forget that I have this course after 5 hours but this is not looking good for PPXL :/
When does the olympics ceremony start?
This one was easy, but there’s a trick here, I first looked this up with Perplexity before the ceremony started, but now I looked it up, while the ceremony already passed, and here, SearchGPT is the clear winner. Why? Well, because both PPXL and Google told me when the ceremony is GOING to be, while SearchGPT correctly surmised that it has already taken place, which is a way more natural way to talk about things that happened!
Not only that, it looks like SearchGPT pulls the event’s photograph (since it already happened!) which makes for a much nicer result
What time did it air where I live?
One of the best things about these new AI searches is the ability to follow up and dig deeper into an answer, and weirdly, even tho I switched to GPT-4 Omni in PPXL, the same exact follow up failed to give me the answer I wanted in PPXL and worked in SearchGPT.
When I tested again, and phrased a bit better (What time did it air where I live?), this worked in PPXL as well ?♂️ Weird, given supposedly it’s the same model underneath? Or maybe OpenAI is running a new model under there??
What was the last episode of ThursdAI about?
This failed in both search engines, while SearchGPT gave an outdated answer, citing a post of mine from June 13 (over a month and a half ago), PPXL just gave me a vague generic answer with no details and no citings and no dates ?
Is this a Substack issue? I tried multiple queries and couldn’t get either search to lead me to the episode I released yesterday ?? At least neither of these try to correct my question to Thursday like Google does ?
Local Results
Both search engines know where I live based on IP, so let’s see how they handle local stuff.
Is it going to rain tomorrow?
Huh? Why does PPXL think I’m in NY here? I was 100% sure that previously it knew where I was? It took me a few seconds to notice that I had the “Pro search” mode one, which apparently puts me in NY!
Once I repeated the search without Pro mode, PPXL found out about where I live, and gave me a nice weather widget, but also was more correct. According to my verification with Apple weather, there’s 0% of rain tomorrow.
I’ll give this one to Perplexity! SearchGPT made me carry an ☂️ for nothing! (who am I kidding, this is Denver, it’s always in the car!)
Weather for the next week
Now that both services found where I am, asking for a weather report for the whole week, shows that SearchGPT has a nice weekly weather widget, very clear and clean, while PPXL, what, I don’t even ?♂️ The graph is still there, but the description says it has no data!
Complex and Multistep queries
This is the area where LLM based searches supposed to shine compared to a regular 10 links based Google like search, especially once they do some advanced processing in the background. Here I will say, Perplexity has a “pro” mode that I absolutely love, that takes longer, breaks down our search into multiple queries and then responds with a great summary, but that takes time so I don’t often do that. For sake of comparison I will try with it both on and off.
What's more expensive, the DMNS or Childrens Museum family plus annual package?
DMNS is Denver Museum of Nature and Science, this is an actual question I wanted answers to, requires visiting two websites, reading and doing some comparisons.
Weirdly here, SearchGPT went lazy on me and didn’t give me the Children’s Museum annual price this time, while it absolutely did give it to me while recording the video ? and answered my question correctly. This shows that both PPXL and SearchGPT often non deterministic results to the same exact queries within a span of around 5 hours! It’s not a shock to me given how LLMs work, but still something to note.
Winner here : Perplexity
What will be the temperature in Hawaii when it'll be the first day of school in Denver?
I don’t know how to judge this one, I think both are wrong-ish. While PPXL did give the correct date of start of school year, SearchGPT just estimated it (again, in the video it estimated correctly, here in screenshot it did NOT), PPXL then proceeded to estimate the weather while SearchGPT tried to at least go and find the weather during this date.
Maybe this one was too complex and needs something like the Pro mode in PPXL ?
How to make the best Shakshuka in the world?
I much prefer the way SearchGPT presents recipes TBH, they are extracting an image that fits the exact step they are describing and it’s really cool. Perplexity shows instructions in a very detailed way.
I will fail both search engines here as someone who knows Shakshuka, both recommended canned tomatoes, and for the BEST shakshuka, you need fresh ones as everyone knows ?
But UI wise, I think SearchGPT takes this one
Multilinguality
LLMs can speak a bunch of languages, let’s see if the search engines can also handle queries and results in different languages
What happened in the opening of the Olympic games (in Hebrew)
Both searches handled this query well, with PPXL giving way more details. One thing to mention also, PPXL shows the Hebrew in RTL (Right to Left) aligned format, while SearchGPT doesn’t.
PPXL gets the win here 100%
How much faster is it to fly to Croatia from Denver than to Hawaii (Russian)
This one is a tricky question, because it’s not faster, flying to Croatia is actually slower, but I wanted to see how the answers will be, PPXL takes this one, as there are no direct flights between Denver and Croatia which it rightly notes!
Neither has issues with the Russian here.
Widgets and Share-ability
I haven’t discovered all the widgets that both PPXL and SearchGPT have, so hard to compare, and it’s obvious that the more mature product will have tons more, but they at least have multi media and images showing for some results, weather related widgets and more
The one thing I like about SearchGPT is the highlighted short answer at the top, while PPXL repeats the query twice (you can see it at the top and then again as the main header on the page), SearchGPT shows your query in grey and the answer in black
PPXL has an amazing shareability option, which I absolutely love, the social media cards are incredible, sharing with friends is a delight, not to mention turning search results into pages. Meanwhile, SearchGPT results cannot be shared (makes sense given this is a limited gated preview)
Should I put glue on pizza?
I had to try and see if the infamous pizza glue incident will happen with these and nope, not only do both of these search engines warn me that it’s not the right thing to do, SearchGPT even throws shade at Google ?
In Summary:
SearchGPT is a great addition to the chatGPT family of products and features, it’s not easy to build a search engine, and even harder to build one from scratch, and even double harder to build one that doesn’t lead to embarrassing examples that will eventually make your company to roll back these features!
This is quite an incredible release from OpenAI given that this isn’t the main focus of their company, and while it’s lacking in features still, and sometimes hallucinates, it’s already very useful, super fast and quite awesome to use, and even at this early stage has some advantages over established competitors!
Will I be switching away form PPXL? Maybe for a bit, I like to test drive things to the fullest, but I’ll probably won’t be using this as my default search for long, I LOVE PPXL PRO ?
Hope you enjoyed this deep dive, see you next ThursdAI
