AI IRL Podcast Episode 48: The Ways AI and Humans Work Together to Compile Knowledge
Twenty years of Google and Wikipedia, but the information is still not at our fingertips.
What if we could automate the collection of information using both AI and humans? And what if we could do it by indexing and creating a better entity map?
On this episode, I interview Jude Gomil, Founder and CEO of Golden, the world’s first self-constructing knowledge database built by artificial and human intelligence. We talked about how to think about, compile, and organize knowledge using both AI and humans.
“One of the things that’s been bugging me when I’ve been searching for things,” Jude said, “is this: I’ve not found the information. It’s been missing, scattered over the web, or fragmented in lots of different areas.”
There’s so much information out there, but we default to Google. And yet…
It’s not sufficient.
It’s not in-depth enough.
It’s not connected enough.
What if instead of searching, we could:
- Build an index of human-readable information and machine-readable information
- Automate the collection of information
- Get good at topic prediction
- And let humans fill the gaps left by AI?
“Up until the last 20 years we’ve had Google and we’ve had Wikipedia, and the information is still not all at our fingertips. It’s not sufficient, it’s not in-depth enough, it’s not connected enough.” —Jude Gomila
Making information commercially available
Golden is not just indexing and creating a better entity map. It’s also making something that the end user can consume in a much easier way.
“It’s important,” Jude said, “because it leads Golden to being commercially available for programmatic access or bulk access to the information than they pay us for.”
Paying customers mean Golden can feed the engine — getting more information and opening up pages for people to be able to read the basic information on the topic.
The main push now is: can we automate the collection of information? Can we make it easier with emerging technology?
“This is a technology play, where we believe we can actually use an algorithm to construct the information with humans in the loop,” Jude said, “and eventually automate all the boring parts of the collection.”
Over time, they could shift the humans over to the fun part, which is the learning components and personal opinions.
“We’re trying to get the boring parts done,” Jude said, “so that humans can spend their time on the interesting parts.”
“We’ve tried to turn the system into a modular problem that we’re conquering piece by piece, and we’ve tried to accelerate it with the combination of AI, prediction, and humans in the loop, with some UI to sign off on the predictions.” —Jude Gomila
The special sauce of information and AI
“Some of the special sauce is actually just stringing together lots of different components,” Jude said, “and then saying, ‘Oh, we really need to be good at that!”
Things like topic prediction and structured data extraction.
With extraction, for example, there are various layers to getting information out of a public document or webpage, and there’s various parts of reading for it and trying to predict what topic is inside there.
Golden has to ask questions for taxonomic prediction:
- What is the object of interest?
- Could it be a new topic for Golden?
- Is it a company?
- Is it a person?
And then once you know what it is, well, Golden has the schema for companies, people, and technologies.
They can then say, for example, “Hey, it’s a company!”
The company must have a CEO, right? Or a location? So can we use a predictor?
“The good news about this,” Jude said, “is it turns it into a modular problem. It turns the collection effort into a series of data funnels, of doing various processes.”
And then of course, there are humans in the loop, too. They sign off on predictions, which means you can collect data from human annotation, and improve it, too.
“Can we automate the collection of information? Can we make it easier with technology that is emerging, for example, with AI?” —Jude Gomila
How to hire a team for AI-driven information collection
“I think it’s mostly a hiring game,” Jude said, “because we’re not going to wait for it to occur. We are going to hire.”
Golden is hiring people in the Bay Area right now in fact. “If you want to, come join us. You get a summarization of structured data extraction.”
That said, it doesn’t have to be a dichotomy.
You can have humans update their profile (or profiles that they know information about) and correct things that the AI may be predicting.
“We obviously want to hire more people where they are experienced,” Jude said, “and we want to leverage anything that may be open source.”
But Golden also wants a distributed set of people to help collect the information, as well, with the unwritten contract that this text is available on creative commons.
“I want this page to be open,” Jude told me. “I want people to look up, for instance, ‘Which mathematicians are working on the Riemann hypothesis?'”
That makes it cool!