As part of its ongoing effort to develop an open search engine that will compete head-on with the likes of Google, Wikia has purchased the Grub Web crawler tool and released it under an open source license, the company announced Friday.
Wikia, founded in 2004 by Wikipedia founder Jimmy Wales along with Angela Beesley, announced the open search project at the end of last year, with the aim of building “a new search platform founded on open source search protocols and human collaboration.”
Because Grub is a Web crawler that operates on a distributed model in which users donate personal computing time when they are not using their machines, one result of the purchase is that Wikia won’t have to develop its own computer network to crawl the Web.
Key Piece of the Puzzle
“One of the key pieces of the puzzle for a full-scale search engine is some method of crawling the Web,” Wales, who serves as Wikia’s chairman, told LinuxInsider.
Originally developed as an open source product, Grub was purchased by LookSmart and made proprietary four years ago; Wikia’s acquisition makes the crawler open source once again.
By combining Grub, which is building a massive, distributed user-contributed processing network, with the power of a wiki to form social consensus, the open source Search Wikia project moves another step closer to a future in which search is open and transparent.
‘Tremendous Response’
Although Wales founded both Wikipedia, the collaborative, online encyclopedia that’s run by the nonprofit Wikimedia Foundation, and San Mateo, Calif.-based Wikia, which is a for-profit company, the two organizations have no formal ties.
In May, Wikia recruited Jeremie Miller, founder of open instant messaging platform Jabber, to head the open search project.
“We’ve had a tremendous response from very interesting commercial players in the search space,” said Wales.
“The desire to collaborate and support a transparent and open platform for search is clearly deeply exciting to both open source and businesses,” he added. “Look for other exciting announcements in the coming months as we collectively work to free the judgment of information from invisible rules inside an algorithmic black box,” Wales said.
An Open Future
“In looking at the overarching industry, it has become clear that open is the business model of the future,” said Michael Grubb, senior vice president of technology and chief technology officer with LookSmart.
“We are pleased to collaborate with Wikia and believe that Grub will thrive under an open source license,” Grubb added. “We are happy to be able to assist in the movement to make search a more open proposition and look forward to seeing things progress from here.”
The acquisition “is a good indication that the project is moving forward,” Greg Sterling, founder of Sterling Market Intelligence, told LinuxInsider. “With its distributed model and democratic nature, Grub is almost a metaphor for the whole project,” he added.
Fostering Innovation
“It’s great that they’ve released Grub as open source again,” agreed Rebecca Wettemann, vice president of Nucleus Research.
“Open source is a great opportunity to drive collaboration and innovation, and that’s where we see some of the best applications come together,” Wettemann told LinuxInsider. “This will enable broader access for innovation in the open source search engine market.”
While Wikia aims to provide an open source alternative to Google, Yahoo and the other major search engines, few expect that to happen anytime soon. Indeed, work is currently under way in the design process on both the social Web site and back-end components, Wales said.
Ambitious Plans
The product is planned for release by the end of this year, “but don’t expect a fully functional product at that point,” Wales cautioned. “There were only three articles in Wikipedia when we launched it,” he explained. “It wasn’t much of an encyclopedia to start, and this won’t be much of a search engine at first.”
Even once it is up and running, Wikia “is not going to overtake Google overnight,” Wettemann said. In addition, “Google is building tremendously right now, far beyond its current search base,” she explained. “Looking just at search is limited view.”
“In the short term, they’re probably not going to have much of an impact,” agreed Sterling. “In the longer term — three, five, seven years away — it’s anybody’s guess.”
‘David vs. Goliath’
With its positioning as a democratic search engine, the product could gain a following among influencers, Sterling said. To make it in the mainstream, though, it will have to work as well as or better than Google, in addition to having some extra feature to differentiate it and get people to use it, Sterling said. “That extra value could be the human layer,” he added.
The open search project welcomes contributors for assistance with programming, testing and feedback, Wales said. “We’re hoping to get some excitement going,” he explained. With its ambitious goals and competitors, “this is a fun project,” he added. “It’s a sort of David vs. Goliath.”
Loading ...
