Andrew Joiner, CEO of Hyperscience, on theCUBE Research: Google Cloud NEXT Preview

Scott Hebner: Enterprise AI is entering a new phase and the challenge is no longer the model. It’s operationalizing inference at scale and doing so in a way that can be trusted in consequential business settings.

Welcome back to the cube. I’m Scott Hebner, principal analyst for AI at the Cube Research. In this conversation, we’ll explore what Hyperscience calls the inference inflection point, the shift from experimentation to production systems that must reason, act, and deliver real business outcomes.

In our conversation today with Andrew Joiner, the CEO of Hyperscience, we’ll break down why operationalizing inference is becoming the central challenge and why trusted data is emerging as the critical constraint and what it really takes to move from AI that talks to AI that can be reliably operated with inside, you know, enterprise workflows. Hello Andrew. Welcome. Thanks so much for taking time to join us today.

Andrew Joiner: Scott, thanks for having me. I’ve got to get down here more often. The last 12 18 months seems like we need a new coat of paint on everything we’ve been sharing. So, happy to be on today.

Scott: That’s great. That’s great. And actually before we dive in, just for our audience, why don’t you give us a quick background on yourself and how you describe Hyperscience and its role in enterprise AI and you know the problems that you’re trying to help solve with your clients.

Andrew: Sure. So, I founded a company. I’m a product entrepreneur back in the early 2000s and like all software companies if you do well you sort of get acquired and the lineage between all of the companies that I’ve been a part of have really been around unstructured data and I’ve been a part of growth-oriented enterprises that have really been typically at an intersection where there have been a collision of forces whether it’s regulation and eDiscovery with types of unstructured data or phone calls and understanding information out of unstructured data for the past 10 years as a CEO really been trying to scale what I call really differentiated technology and Hyperscience sits in the middle of that sort of vision.

It’s got very differentiated tech founded by preeminent ML scientists to really read documents while most of AI gets excited around the generative portion of generating information. What we focused on is reading information at scale using AI to really help organizations make faster decisions or really get ready for the future of AI really take this information that’s locked in documents and feed the models and feed the AI that wants to reason over that data in the future.

Scott: Yeah, so when you talk quite a bit about the shift from AI experimentation to production scale inference in particular. So define that particularly for business leaders that may be listening in what actually is changing in the enterprise as this all occurs that shift

Andrew: In the early days. I think the biggest struggle with AI was getting real proven ROI and I think everyone could naturally see the power of the tool of the models. The models are incredibly powerful and certainly as individuals at the edge we get massive leaps in productivity whether it’s writing or summarizing the work we do all day but getting this operationalized in the core of business processes where what the processes that really run our economy.

You know if you think about applying for a mortgage you typically submit a form you apply for it and then you submit your supporting information. If you’re applying for like a benefit, like a SNAP benefit for nutritional supplements, you fill out a form, then you justify your income, and then the government reviews that and approves it. These processes are at the core of our economy, but they’ve eluded AI.

But what we’ve been working on is operationalizing this, getting this into production so that these decisions happen more quickly. At the Veteran Affairs Association, it’s one of the best stories we have. It used to take three months for a veteran’s claims to be adjudicated. But using AI, that speed is now down to about three days. That’s because a human dependency on reviewing these historical records can be done accurately by AI so we get faster decisions. And that’s the type of production scale we want to offer to a broader subset of the market.

Scott: Yeah, we run an ongoing survey called the Agentic AI futures index and one of the things that showed in the last iteration of it was only 49% of business leaders in the enterprise actually trust generative AI on its own to help make decisions and and at the same time 79% were looking to invest even more in creating what I guess they would call trust frameworks and I can also tell you just from all the conversations I get to have with different companies in the enterprise AI marketplace. If there’s one word I would use to describe the state of enterprise AI, it’s the word trust. And I think that’s what you’re hitting on, right?

Andrew: Well, there’s two facets to trust in my mind. So, the first trust is can you trust that we’re reading the data accurately? You know, a mortgage is a big decision for a family and you want to make sure you review it accurately, you do it with trust so that all of it’s fair, transparent, auditable. These are big decisions regardless of whether you’re paying someone, whether you’re borrowing money or making an adjudication claim.

But the other side of it is you’re using these technologies that are very powerful. So how is my enterprise data being used? And from the beginning, Hyperscience knew that this would be a fundamental challenge. Like all enterprise technologies, if you’re going to go deep in the enterprise, you got to solve security, you got to solve compliance, you got to solve sovereignty, you have to solve these hard security issues.

So we allow clients to run on premises in private cloud. We allow you to train on your own data. You build your own models. You control your own flows. And so we operate at the highest levels of security. We’re at Fed Ramp High. We run IL5. We can run airgapped environments. And you have to sort of clear those barriers if you will so that clients can trust running their core business processes on this type of powerful technology.

Scott: Yeah. And I think the idea of an LLM only architecture without all these extra layers and guard rails and things of that nature will only get us so far in the enterprise. And I think that’s part of the constraint on ROI. Certainly individual productivity has gone up, but when you start looking at teams and processes and workflows, I think that’s what you’re getting at here. It’s going to take a lot more than what most enterprises have in place today.

Andrew: Well, you want to use these powerful frontier models. It has to be a tool in your core architecture. But doing that is hard. And so that’s why we think we’ve reached what we call the inference inflection point. And what we mean by that is it’s not just one tool for all the tasks that you need. The LLMs, there’s going to be a scarcity of tokens. There’s going to be an affordability crisis. As these tools start to come into your P&L and are charged for them.

And so as we look at trying to drive, we have to drive high accuracy for our clients. We have to be accurate on your paystubs, on your forms, on all your unstructured data that you’re submitting. We want it to be automated. Otherwise, why bring it into the enterprise? But it has to be affordable. And so what we’ve essentially entered into is an inference layered optimization approach. And what we mean by that is we use the right inference for the right task.

So, when you get a box of documents and you submit a mortgage application, as we’ve all probably gone through, there’s probably 20 or so documents that you attach, your assets, your income, your residency, your home inspection report. Understanding what those documents are is a hard inference task. Now, you could use an LLM to do that, but these are dense, long documents, and you use up a lot of tokens for a very simple task. So, there’s smarter, more specific inference AI that you can use to solve those types of tasks.

What we like to use the LLMs for is the hardest reasoning task. So, let’s say someone has a gap in their income and they’re trying to essentially make sure you have the income coverage, but you need to read the letter to understand what that inference needs to be. So, we use LLM for reasoning. We use them for very hard inference task, but it’s only on a portion of the overall tasks that flow through the business process. When you’re scaling to billions of documents, which many of these companies are operating at when you look at insurance, financial services or government agencies, you have to be thoughtful about the inference you’re using.

Scott: So, help put this into the context of AI factory, which I know you have been, I think, framing this as, right? Like what does it actually look like operationally inside a real enterprise environment?

Andrew: Well, essentially what you think about is a data center becomes an AI factory. And what that means is one of the primary capabilities they unleash is the provisioning of tokens to do inference. So in our task, most of the business processes that organizations run on today have been outsourced. They’re done by BPO organizations. It’s difficult tasks that have eluded computers for a very long time because there’s so many policies and things that have to be adjudicated when reading documents.

So they’ve outsourced that to human keyers and human keyers key in the data or they reason over the data to help feed those decision systems. Well, those are going to become more and more inference related tasks. Now we can look for and use inference to determine whether an invoice has a purchase order or whether it’s backdated or whether it has fraud. Those used to be cognitive tasks that you would rely on humans. And by the way, they’re tedious and they’re not very liked, but they were outsourced to save money.

In the future, this looks like an inference task. And so these AI factories are unlocking this cognitive capability. And when it goes agentic which is really the next step and it’s coming very quickly meaning the entire business process is orchestrated by layers of inference the data factory and the AI factory now allows organizations to run that in the way that they need to at the scale of the inference that these organizations need and it’s a massive amount of compute that is needed.

Scott: Right and that’s when you start you start to handle all that and harness it that’s when you start to ramp up that ROI.

Andrew: You want to ramp up the ROI if you make it affordable. And that’s why you have to be thoughtful about the inference you use. The models are powerful and you can use a powerful model for a simple task, but that essentially eats up your allotment of use at the AI factory. Ultimately, AI factories are limited by physical compute. So, you’ve got to be thoughtful about how you optimize your inference and that also aligns to your ROI. When you do all of that, you will absolutely get tremendous ROI benefit.

Scott: Right. It’s all headed in the right direction. Well, you also emphasize that, you know, the real bottleneck is not necessarily in the model, but it’s in the data. And with that, why does ground truth, as you say, become the limiting factor here in terms of scaling all this?

Andrew: Well, when we got into the market, what we showed with our inference is that we could get to decisions more quickly. So we talked about the veteran affairs that was a wonderful case study for our veterans. We all care about that constituency and the fact that their claims are not getting gummed up in the system for over 3 months into 3 days is real ROI. So that’s really positive.

But when you have to make a decision at the end of the day you’re only looking at about 10 to 15 critical data points that you’ve essentially got to validate cross reference to make sure that that decision is made properly in the future of AI. If you want AI also helping you reason and make those decisions, it needs a lot more context. So what was a difficult task that still proved ROI is now about to become to a level of industrial production. You now need about two to 300 data points. You may be adding data to those documents to help the agents do reasoning to make sure we’re doing those decisions correctly.

So the order of magnitude, what plagued the industry, it’s been plaguing for 20 years to get 10 to 15 data points quickly. I’ll give you an example. The SNAP administration, so it’s administered by all the states. It puts food on the tables. Everyone has a little grace because it’s a hard task. They get a 6% error rate. 44 states fail that process. And more than 40% of SNAP applicants when they submit their documentation are kicked out in the first 30 days.

Now, that’s money that could go back into the economy. And that’s because we’re relying on processes that have a human dependence. And it’s just really hard to do that at scale. The average error rate is about 15%. So that’s dollar for dollar you could claw back from the states. So, we need AI to help us. So, we can do that and lower the payment error rate, get more money back into the county, get more money getting approved for these SNAP SNAP applicants, but we can really speed this up for the states and deliver an effective government by allowing agents to really orchestrate what is an assembly of about 20 or so documents for a SNAP applicant. Orchestrate essentially that whole process of getting it right with the applicant and getting it right for the state. And so that order of magnitude is what’s coming and Hyperscience is there to help organizations or governments and states essentially bridge that divide.

Scott: Yeah. So when you’re looking back at the legacy approaches that many of these organizations have had in place. That’s what you’re trying to address the key inefficiencies of that and help them make that leap forward. Right. And those inefficiencies are the data, if I understand it correctly, the quality of the data, the inference, and being able to apply your tokens and your compute power for the things that matter the most. Is that fair?

Andrew: Yes. At the Veteran Affairs, there’s over 25,000 forms that they have to process. So that’s just a wide range of information that flows into an organization. And you can imagine as a key or someone reading against all these different forms, it’s a really hard task. So it’s just like a classic unstructured data problem. There’s high variety. It’s high volume. It tends to be dense and you have to be accurate.

So it’s a hard technical problem at the beginning, but now you have to add in the fact that you have high security standards. You have high governance and trust needs. You have to essentially comply with regulations and policy of the organizations. And there’s people at the other end that are really depending on this decision. You either get paid, you get approved for a benefit, you get approved for a mortgage. So, it’s a high stakes, high impact sort of situation where the cost of failure is high.

And you’re right, getting trusted data is the key, but having auditability and transparency is critical. And now you got to do it at scale. And so, this is one of the hardest AI problems I think that exists out there. And why you need dedicated infrastructure. This is not solved by an application. You need a doc ops infrastructure that allows you to layer inference, monitor your cost, do so with transparency, handle logic and that’s what these platforms like Hyperscience are doing.

Scott: So define your notion of a dual loop model. How do the enterprises think about, I think what you’re saying is separating high volume automation from the higher order reasoning systems and that’s is that the dual loop?

Andrew: It is. So the way we try to explain it with customers is the top loop is typically your automation loop. This helps you deliver better experiences to your citizens, to your veterans, to your claimants, to your borrowers because you get decisions more quickly or if your information is not in good order, you get essentially notified more quickly. That’s the automation loop. And organizations can get proven ROI by going on that AI journey.

But at the end of the day, what we’re really extracting is we’re putting that information into systems of record. Your CRM systems, your ERP systems, where you pay people, where you approve things. And so that top loop at the end of the day only requires about 10 to 15 points of data to essentially make sure that decision. Now, that’s what happened. The bottom loop though is allowing AI to start helping you with the reasoning and all of the pathways that this information needs to flow through for approval. It can be done a lot better and I think the promise of Agentic is going to unlock that but organizations aren’t there yet.

They’re still trying to make sure they can get this into their core of the enterprise and just drive automation. But with the pace of the market, the bottom loop will become essential because that’s going to be the competitive advantage. If you can underwrite mortgages, if you can approve claims more quickly, if you’re able to have faster freight finance and prove truckers more quickly, it’s going to be a competitive advantage. And Agentic is going to unlock that. But it is an order of magnitude more difficult.

And that’s the journey we’re trying to showcase that if you want to futureproof your investment, if you were to buy a legacy IDP technology today, it may not be able to solve really the upper loop all that well, but it is certainly illequipped for the bottom loop where Gen AI is going to essentially allow you to reason and take over these processes.

Scott: Yeah, one of one of the other findings in our Agentic AI futures index was over the next 18 months 62% of the respondents said that they’re investing to move from a more pure automation oriented objective to knowledge work based in reasoning and having to make judgments and things of that nature and I think that’s that higher order reasoning that you’re talking about right?

Andrew: It is. You know our application is essentially a pipeline kind of cognitive task underpinned by Python. And I think what’s really dawning on the software market at large is the ability for the models to write formative code and code that can actually be deployed reasonably into production. It’s still quite difficult. You still have to understand how to release and develop software. But with that, our platform capabilities are really getting unlocked.

And I think it’s unlocking a lot of different knowledge tasks. Now we have explainability as you’re pulling up a document. We can essentially explain how the LLM is forming the opinion. If currency isn’t on the page, but it identifies more likely what the currency is. We can give that kind of explainability. And you need to do those types of tasks in code. You can’t write rules. You’ve got to have inference. You’ve got to layer it in. And so I agree with you like I think that there’s a lot of agent washing.

But I think people now are starting to realize that with code agents, going headless, being able to access things without the restriction of an application interface is going to unlock an amazing set of productivity. And now the task for some organizations like us is to get it into the core of the enterprise and others may focus on other areas like sales and marketing tasks. But for us, it’s unlocking these really core mission critical high cost of failure processes that I believe have eluded AI but are about to be unlocked by AI.

Scott: Yeah, you mentioned explainability, something I’m really passionate about. Which kind of brings me to a question here around governance and control. So as these organizations move increasingly towards you know a more true agentic AI posture, what what has to be put in place to help the users trust these systems and like in real workflows because I think as you pointed out with the explainability comment, you know, in the end the people that are using this technology need to trust it.

Andrew: Well, from the beginning we had a concept built into our platform called human in the loop. And what human in the loop allowed the system to orchestrate itself is when it wasn’t confident it could raise its hand. And so it’s this beautiful symbiosis between the model reasoning as hard as it can and then it letting you know essentially when it’s not confident that it has its answer correct.

Now what we do is most organizations want the maximum ROI. They don’t want to put humans in the loop but they do know they need that governance. They need that trust. They need that explainability. They need that oversight. What we’re able to do now is put AI in the loop and use consensus or double blind by multiple models to essentially when they agree, essentially pass that along.

But you always have now what we call humans on the loop. And humans on the loop to me allow you to oversee the entire process. There’s so many inference decisions that are being made. It’s hard to go down and look at every single individual decision. But now you can look at it in aggregate and use models to help you do that.

So I believe we’re moving more towards an AI in the loop where you use multiple models and consensus but have humans on the loop so that the explainability, the transparency, the governance, you can hold these companies to task in these difficult documents. They do hallucinate. They will make bad decisions. And so you do need humans on the loop to essentially make sure that what you’re doing is accurate. But by the way, humans make mistakes, too. It’s never going to be perfect. And so you need a system that has the right auditability, traceability to allow you to do this at scale within reason.

Scott: Yeah, I love that term human on the loop. That great. Yeah, because you know in the end in the enterprise just because the model told me to do so isn’t an audit trail. So you need more, right? And I think that’s exactly what you’re explaining here.

Andrew: Yeah, what you want to do is you want to use these models but harness them in the best way you can. So I like the idea of taking two independent inferences. We have a concept called vibe which is a really neat concept. And what that concept is is it allows you to take two independent inferences.

Let’s say you have something difficult like a bank statement of which there’s debits and credits. Well, what we can do is we can use one model to look at just the total line. it can just spot the totals between each statement or each of the tables. And then another model will look at the details of the table, all the line item details. And what we’ll do is we’ll sum up the line item details to make sure it matches the spot of the other total. So it’s two independent inferences that we use to make sure we’ve calculated it correctly.

And when we do, we can pass along like a green bounding box down to other people in the organization to say, “We’ve read this statement and now it can be trusted. It’s in the green.” And if it’s not to be trusted, we can call it red and have someone review it because potentially it’s a malicious bank statement. And so we have those types of techniques when you’re processing documents at scale whereby there’s form changes, there’s checks where we can do these type of comparative inference calls to essentially give us high confidence that what we’re reading is trusted, correct, and explainable.

Scott: Yeah, that’s some really really key stuff. As I said before, just all my interactions and what I’m seeing in the survey data is that one word right now is trust. And it looks like you’re surrounding it from multiple facets to really bring it to life, right? And I think that gets solved or at least enterprises become more and more comfortable with the trust issue. That’s where you’re going to start seeing the ramp of ROI. So you’re

Andrew: I think there’s three things, right, Scott? You got, you know, accuracy, you got automation. You can be accurate with humans, but you’re not very automated. It’s not very affordable. So you need accuracy, you need automation, but you need affordability. But as you said, it’s got to all be underpinned by security, trust, and governance. And I think there’s a number of companies out there that are really partnering with the model providers to deliver that into the enterprise.

And you know this, every enterprise technology runs into that. I’ve been through them all. Been through web, cloud, mobile, and every time, you know, you couldn’t even bring your own device to work because it wasn’t trusted. So, we get through these things. It is hard work, but that’s the journey we’re on and that’s what we’re excited about at Google Next.

So, we’re going to be there this week really showcasing some of the power of Gemini inside of our platform, using it for AI in the loop, using it for really difficult reasoning tasks alongside the Hyperscience inference, but we’re going to show how we can do it inside of GCP, the Google Cloud, essentially with top security, top compliance, top transparency.

So, we call it, we’re kind of playing on a music theme, the DocStars. I don’t know, Scott, maybe we’re out of good ideas, but I think it’ll be a good one. So, we’re going to play on that theme. We’re going to unleash the Doc Stars. We got some cool album covers, but that’s what we’re going to be showcasing at Google Next this week.

Scott: Yes. Keep cranking forward. I think you’re absolutely on the right, you’re on the right messages here that I think are really going to resonate and and ultimately is critical for these enterprises. And before we wrap, Andrew, tell us a little bit about what you’re doing at Google Next and where people can find you and perhaps, you know, one piece of advice for the audience listening in, particularly the leaders on what should they do next.

Andrew: So I believe you have to lean into now using AI and the core of your enterprise. We’ve been essentially showcasing the power of the tools at the edges of the enterprise with individual productivity. I think we’ve moved into the development and engineering parts of the organization and it’s showcasing itself to be quite powerful for a lot of tech companies essentially but we’ve now got to harness that into the core of the enterprise and I believe Hyperscience’s infrastructure plays a role a role in unlocking that so we’re showcasing our latest release of the software we posted a blog we’re showcasing the upper loop and lower loop so we’re showcasing for clients how you can get stunningly good results at reading complex information across the enterprise, help you make faster decisions on the upper loop, but also get ready for Gen AI, which allows you to query and reason over the information that you’ve really never had access to because it’s been trapped.

So, that’s going to be our showcase. We have demos and if I was going to ask anyone to come by, come by, see the infrastructure, see the power of it, but get comforted by the fact that it’s done right, that we do it with the right security, humans on the loop at all cases when need be, so that you can safely bring this and tackle these core processes that have eluded people for 20 plus years.

Scott: And for those of you in the audience that aren’t going to be at Google Next, make sure you go to hyperscience.ai to learn more. Andrew, this has been a great conversation. I think very illustrative of where we all need to move here and a lot of great insight. So I really appreciate you taking the time to be here today

Andrew: Scott, always appreciate the opportunity to be here. So thank you.

Scott: You bet. And I think what you know what comes clearly as the enterprise AI moves beyond experimentation into this real world execution is you know success will be defined by operationalizing inference at scale. And as we just discussed with Andrew, that ultimately comes down to whether organizations can trust the data and the decisions and the systems themselves. So, thanks so much for sharing your perspective again, Andrew, and what it’s really going to take to move the industry forward here. And for all of you, thank you for watching the Cube. We are the leader in enterprise tech news and analysis. We’ll see you again soon