Everybody uses search engines in this day in age whether it be google, yahoo or even bing. Search engines are used to find a song based on a single lyric, to look for articles in certain field of study, or even to search for online mini games. Sometimes they can be a bit inaccurate and what you are trying to find can end up on page 3 of seemingly infinite results, and nobody goes to page 3. Next, the algorithm notices what website you clicked on in relation to what you typed so that the algorithm can effectively improve upon itself.
Search engines work by determining the relationship between the words that are typed and then algorithms find those same words in a plethora of websites. If your words show up more frequently in website A than website B, then website A will be higher on the list of results than website B. It’s a simple process in the minds of computer science but very few people actually give it thought.
The famous engines however are not as smart as we think since they lack language understanding as well as logic. They may also end up bringing us old news rather than new information when we type the same thing in the search bar essentially meaning that it deepens biases by thinking that if you type the same words, then you want the same website. Naturally, this may not always be the case. Researchers noticed this weakness and immediately wanted to strengthen it.
Matthew Lease, an associate professor in the School of Information at The University of Texas at Austin, has discovered that by combining two very powerful components in the computer world, they can potentially improve search engines. “What are these two features?” you may be asking yourself. Artificial Intelligence (especially that of natural language processors), and crowdsourcing. Lease believes that these two components may lead to better information retrieval (IR) systems in search engines. He intends to not only improve general search engines but also engines like google scholar and JSTOR in order to allow you to find a specific study without using up valuable time and effort.
"An important challenge in natural language processing is accurately finding important information contained in free-text, which lets us extract into databases and combine it with other data in order to make more intelligent decisions and new discoveries," Lease said. "We've been using crowdsourcing to annotate medical and news articles at scale so that our intelligent systems will be able to more accurately find the key information contained in each article."
Lease found that his method was able to train a neural network (a form of AI modeled on the human nervous system) so the network could accurately predict named entities and extract relevant information. The new method improved upon existing tagging and training methods.
Lease’s team also applied a form of weight sharing (where words that are similar share some fraction of a weight and are assigned a numerical value to find a website that matches your search) to a sentiment analysis of movie reviews and to a biomedical search related to anemia. This approach consistently led to improved performance and accuracy on classification tasks compared to strategies that did not use weight sharing.
Simply put, the new and improved search engines will more accurately predict what you are looking for through AI, crowdsourcing and, on a lesser note, weight sharing. This is something we will all appreciate.