We all create a lot of data to process. It's not a secret that our footprint is growing pretty fast. Logs, files, media, settings, history of your Facebook comments. Everything can be useful if correctly analyzed and used.
But we had a lot of data before this web thing. We have been literally sitting on 50-60 years of numbers. Even if each year we grow it exponentially, sooner or later, those digits will be very useful for business. Yep, business and government will be the first who will utilize it.
And this growth of data is a big problem. How can we organize and take advantage of it?
"Data Is The New Oil."
Ok, I get it. The value is enormous. But with great power...
How can we access this data correctly? Without simplifying it or opening this door, information is just bits.
It's like having a big shelf of books but using them only as a fancy decoration to your home.
Searches are everywhere and have so many different options.
And context is crucial as well. Even having cool products like Elasticsearch still can't cover 100% of this Problem. Why? Because it's very complicated. The more information we have, the more difficult it becomes to deliver it with the speed that we want that data. And it's never a lot of speed for humanity.
We Always Need More!
Imagine that Elon will connect us to the Web, but when you google something using your neurons and you see loading.gif or billions of results.
NLP has a Big Problem. And it's a Search. This is why big boys control it. Let's put it differently. Only ppl that can afford a private submarine, can do a search. Ok, James Cameron didn't build search (but I'll put a joke about Skynet).
I was talking about FAAGM having complex searches, that we use each day. But even they are failing. I'm talking about Linkedin search. It's awful. But it makes me happy when I see how big players have issues with code. It means that I still have a chance. Search is a billion-dollar business place. And we'll see new players, dominating the market pretty soon. And I think I need to look for products from the Asia region. They are just growing quicker and have a lot of ambitions.
And to get more correct results, we need to teach a machine to understand us better. We have a lot of progress there, but because we are still making money from redesigning or upgrading websites, there’s still not a lot of progress that can satisfy us.
I mean fak, Google has been looking the same for 15 years. And like why we need billions of results there? Most times, I'm not jumping from the first page. But I'm interested to see a percentage of how often people google themself or info about their Ex's. There should be a product that prevents you from doing that.
I don't have a solution for you, don't think, but I can tell that building search from scratch is a bad idea.
You don't have so much infrastructure. It's like trying to get oil from the earth, using only your hands. You should use projects that have been doing it for years and have more core value for that.
Data is not only for your smartphone apps. It's a frontier for science. This helps us to get in space or track some pandemics.
