How to Increase Efficiency with Intelligent Automation

Richa Bharwani: Afternoon, and welcome to the Hyperscience Public Sector Webinar: How to Increase Your Department’s Efficiency with Intelligent Automation. My name is Richa Bharwani, Field Marketing Manager at Public Sector. Before I turn it over to our speaker, Todd Pratt, Principal Solutions Engineer, I wanted to let you know if you have any questions, please enter them into the Ask question window located on the left side of your screen. We’ll have Jeff Cahill, Federal Solutions Engineer, answering your questions throughout the webinar. After the demo, both Todd and Jeff will answer some questions live for you as well.

Todd Pratt: Thanks, Richa. Thanks everybody for joining us today. We’re gonna talk a bit about Hyperscience and how you can use the Hyperscience platform to increase the efficiency that you have inside of your different processes with some intelligent automation. We’re really an automation company. Our mission is to connect human and artificial intelligence together to solve automation challenges. We’re doing that at federal and state agencies today by combining state-of-the-art machine learning and some proprietary human-in-the-loop technology that we’ve built into really an easy-to-use platform.

Todd Pratt: That effort has really yielded us some great customers in both the federal and the state arena. Working with state partners or federal system integrators and directly with different agencies, we’ve learned that there’s still a lot of opportunity out there. We’re finding that there’s really some massive volumes of information and business processes that are document-driven that could use some efficiency. We’ve also learned from our customers that accuracy is absolutely paramount. Especially in this industry of document processing, lots of times people would see this sort of a process as a black box. We really have a focus on accuracy. We’ll see that in the application. And then automation. It’s good that the information is accurate, but speed matters to our clients as well so that we can automate some of those difficult or manual processes that are taking up so much time today.

Todd Pratt: I want to talk just a second about sort of the old and current landscape. From a document processing perspective, a lot of things still look the same. The document processing industry has been around for some time. Documents get created in all various sorts of places and then they get sent into a central location or to a central agency via certified mail. As soon as mail comes in, it gets sorted, classified and prepped before we do any paper scanning. We also see electronic documents coming in via email, FTP or web portals or even mobile phones now. All of that still has to go through that same sort of capture and sorting and classification process. Once it enters the organization there’s a lot of manual steps to capture the information. You’d be surprised how much structured information doesn’t go through any sort of character recognition process, especially with governmental forms that have handwriting on ’em simply because it hasn’t existed in a way that you could automate it much today.

Todd Pratt: Hyperscience tries to address all of these problems. We do that by using some artificial intelligence and some machine learning tools. With our intelligent document processing platform, sort of the things that you’ve probably heard before: classification, extraction, validation of information and the enrichment of that information from input to outcome. This is really where we play in a document process, but we play in a new and different way. We’re uniquely building a modern platform that makes use of machine learning and artificial intelligence tools to do the document classification. You’d be surprised that even though you may have classification tools and you’re sorting things out, there’s a whole bunch of documents that just sort of get massed that are lumped into an “other” pile. We actually have tools that can go through those other piles and identify places where you might be able to categorize your documents more effectively. We’re really good at identifying where this information is gonna be on not just structured documents and handwritten documents, but really all sorts of semi-structured documents as well using machine learning techniques.

Todd Pratt: These are really the things that set us apart. It’s that AI and ML platform approach, truly a business platform with AI at the forefront of it, in combination with an easy to use and easy to integrate application. It’s that vision of the combination between artificial intelligence and humans together that come up with the most accurate platform available. We have our own proprietary handwriting extraction engine, and that’s a true handwriting engine, not sort of the ICR (Intelligent Character Recognition) if you’re familiar with the industry, but an open handwriting and cursive extraction engine that is uniquely ours in the industry.

Todd Pratt: Welcome to the demo portion of the webinar. We’re gonna take a little time and talk about all the differentiators that Hyperscience has and then actually show you them inside of the platform. This is truly an enterprise platform. We have customers that are running millions and hundreds of thousands of these documents per week or millions per year. It’s either an on-prem installation or you can put it inside of your own or your customer’s private cloud. Everything you’re gonna see is fully web configured and operated. We integrate with single sign-on to the user’s network security and we also have internal security so that each group or each user only is able to see the different tasks or the different submissions that they have access to see.

Todd Pratt: The Flow Studio is really just a collection of blocks that put together will create a processing flow. Out of the box, the first processing flow that comes with the application is just our normal document processing flow. We’re gonna go through actually each one of these and do a demonstration of our classification and transcription and identification capabilities. These are sort of the standard building blocks for an intelligent document process. You can certainly build and customize your flows for whatever your business process needs. For example, perhaps you just want to create a quick flow that will redact social security numbers or tax identification numbers. In that case, it’s just a couple of small blocks easily called via our API through some larger business process.

Todd Pratt: I’m gonna go to our submission screen and let this start processing. I’m gonna get just a PDF and this PDF contains lots of different images in it, different documents all sort of squished together. And I’m gonna submit that to that document processing flow that we were talking about. As soon as the submission is complete, it’s gonna go into a status of processing because it’s gonna be going through the different blocks of the flow that we indicated. Normally people don’t use it like I did. Normally this is part of some larger business process where people are submitting information via our REST API programmatically. But we can also do things like sweep network folders or we can monitor email addresses. We have connectors to all of the common RPA tools that are out there as well as message queues.

Todd Pratt: When the submission came into the application, the first thing that it did was it’s going through machine classification and then maybe a stop at manual classification. Classification is simply the software identifying what the document types are. It goes page by page using a couple of different machine learning tools to try to do that. For structured documents, meaning documents that are sort of like governmental forms where the boxes really don’t change from one page to the next, it’s really just a matter of providing the software with a blank version of that form. The software learns the thumbprint using some machine learning tools and some computer vision technology. For semi-structured or unstructured documents, meaning documents like pay stubs or invoices where information is somewhere on there, the software will actually read the documents or extract all of the words or segments from the document and use that to build a classification model. We set an output accuracy threshold, and if the software doesn’t meet that threshold, then we’re gonna send it to manual classification for a human to make a judgment call.

Todd Pratt: Let’s go over to our tasks and see if we have anything available. I’m going to just perform the task that’s in front of me. The software has already gone through all of the pages that it was able to identify either with the thumbprint or with that natural language processing and group those documents into one or multiple pages. It’s just raising its hand down here saying, “Hey, this document’s uncategorized, can you help me out with what it is?” The software’s probably never seen this before. It’s some sort of Doctor Who pizza flyer that made its way into a submission. All we want to do is just tell the software that’s not part of the layouts that we care about in this business process.

Todd Pratt: After classification, the software is going to take a minute to perform machine identification. This is another place where we’re applying machine learning to actually identify where on a document or on a semi-structured document a field should be located. For example with pay stubs, you may be looking for the employee name or their gross pay. From pay stub to pay stub, the layout may be very different. By using the software, the operators can actually just tag where those pieces of information are located. As operators use the application, the software is going to start learning where those fields are regardless of the layout of the pay stub. The software just simply by tagging uses sort of a one-shot model. We’re building a true deep neural network model to identify where information is located for each one of the layouts that we’ve configured in the system.

Todd Pratt: We have an identification task waiting for us. So I’m just gonna perform that task. In this case, it’s a driver’s license and the software is saying, “Hey, I need a little help locating the first name.” Well, if we look here, it looks like Avery Joseph should be it. And so it may be incorrectly tagged sample there. So if we look for maybe last name, yep, that’s not a last name. Let’s say that Joseph is the last name. And you know what? Avery’s probably the first name. The rest of the data, the rest of the fields, the software is actually able to locate.

Todd Pratt: We’ve classified and separated the documents. We’ve identified where all the boxes are. And so now the software’s actually transcribing the information that’s located inside of those boxes. This is really where Hyperscience has an advantage over any other document platform out there. We’ve created our own proprietary extraction engine, and that extraction engine has the ability of capturing not just text but also unconstrained hand print or even cursive. We have a few transcription tasks waiting for us. Again as the operator, I’m simply gonna correct or type in where the software brings me. So Calgary here, Thomas Edison, and then 34. And that was it.

Todd Pratt: At this point, we’ve classified the documents, identified where the information should be located, and were able to transcribe the information. Anything that we weren’t able to do automatically, the software had a very quick and intuitive interface for an operator to correct. If I go back and look back at our submissions tab, we can see that the status is complete and then we have 14 documents. Some high level statistics here: of those 14 documents, there were 16 individual pages identified as a document, that one unmatched page, and we captured 365 fields with pretty minimal intervention.

Todd Pratt: Let’s take a look at some of that extraction. If I hop into this claim form here, you can see quite a bit of text information, some of it clear, some of it not so clear. Everything that we captured in this form, on the right hand side you can see the raw extraction by the software engine. And then we will always say who identified where the box was and who transcribed it. In this case, the machine did both. Also importantly, on top of that proprietary engine that we have that can read handwriting and text, we also have data types. It’s probably not surprising that these data types themselves are also machine learned. So we have data types for name and for address and emails and phone numbers, cities and states and zips, as well as just freeform characters.

Todd Pratt: Let’s look at this SSA form. Quality’s not so great here, but again, we’re doing some pretty significant extraction of handwriting. So June 12th, 1987, again we will normalize that. We can see Johnny Rocket here with this name data type captured that accurately. I really like to draw attention to this field because it speaks volumes to the ability of our engine: 111 Illinois Avenue. Because the way that we built our engine is really based off of a semantic understanding of how people write, and we capture things in words, not necessarily characters. When it’s trying to interpret something that’s specific in a box, it really gives us an advantage over a traditional OCR/ICR engine that would try to interpret these things one character at a time. If you interpreted these character by character, it really causes those engines some trouble because the Is look like Ls and the ones all sort of blend together. But because of the way that we’ve trained our application and because of the data types that we can put in place, our software is actually able to capture this relatively difficult capture with high degrees of accuracy.

Todd Pratt: I wanna jump over to just a couple of semi-structured documents. These are documents where the layout varies from one to the other, but because of our field identification machine learning, the software’s able to go and predict where these locations are. Because it’s text, we’re gonna get a really high degree of accuracy and capturing that information. This is just a pay stub, and if we hop down and look at a couple driver’s licenses that are quite different. Found the information here for Nick’s sample from an Arkansas driver’s license. And using the exact same model, this additional driver’s license that we helped correct for Josephs. This is actually where I identified where the box was located for the first name, but the machine was able to transcribe the word for the letters inside of the box, Avery, without me having to intervene.

Todd Pratt: Let’s talk a little bit about accuracy and automation. A real differentiator for Hyperscience is how we approach accuracy and automation. There’s probably three concepts to understand here. One is output accuracy. Output accuracy is what’s actually delivered to your line of business applications from the Hyperscience platform. And so that’s the combination of both the machine and the human together to achieve some sort of very high degree of accuracy. It’s not unsurprising in our application that we have folks set the accuracy to something higher than 98 or 98.5%. Normal data entry operators are somewhere around 98.5% accurate on a first pass. And so that’s usually the starting point for our application.

Todd Pratt: As you demand more accuracy out of the application, then this software’s just gonna have to raise its hand more often and ask for a human to make some judgment calls. So the higher the accuracy, the less the automation. In this case, this is our technical validation instance, and this is real data from a proof of concept that I performed last month where the client wanted to set the target accuracy roughly the same as a human operator. After doing some processing and some quality control to reach ground truth at a target accuracy of 98.5%, the software predicted that we would be able to automate 87% of the fields. To read that another way might be say for every thousand fields that come through here, the software only has the ability to get 15 wrong, roughly the same as a human would get wrong. And while maintaining that output accuracy of 98.5%, we’re going to automate roughly 870 out of every thousand fields.

Todd Pratt: Just to wrap things up and review what we went over today. You can create your own flows specific to your business process. Those flows consist of blocks, and then each of the blocks have different things that they do strung together to create an entire document process. We talked a little bit about how we use different types of machine learning for our classification. We talked about field identification, how we use field models for every layout in the application that improves over time. And then we have our machine transcription. The base of this is our Hyperscience engine, that proprietary engine that allows us to capture both unconstrained hand print or cursive as well as interchangeably with text information. And then probably most importantly, our reporting, our ability to use machine learning to fine tune how the software interprets information so that you can predict not only the level of accuracy but verify the level of automation that we’re going to achieve.

Todd Pratt: We’ll move on to the Q&A section of the webinar. I see that we have at least a couple questions. Jeff, one question we do have coming in is: how many samples are needed for your extraction models?

Todd Pratt: Good question, and it really depends. If it’s a structured document, we really just need one blank form, and the software can interpret the thumbprint based off of just that one blank. If we’re talking about semi-structured documents, then it’s really based off of the variations that you expect to receive. If you have lots of variations, like invoices or pay stubs, our product team says somewhere in the neighborhood of about 400 samples once we’ve reached 400 samples tagged, then the software is going to have probably a pretty good level of automation.

Jeff Cahill: Thank you, Todd. Coming in about barcodes. Why don’t I go ahead and combine two questions into one. How do you configure it to parse out the different formats in the fields? And if you have a document with both a barcode and data in fields, choose between the barcode parse fields and the fields extracted from the form fields based on quality.

Todd Pratt: When it comes to barcodes, yes, we absolutely support the capturing of barcodes. You can overlap fields so you can look for it. So if you have the barcode value sort of written out like the human eye can see, and then we have the barcode value there itself, we can capture those in two separate steps. If the barcode is distorted for some reason, you have the chance of actually catching the verbiage that’s below or right next to the barcode. And then using the platform, we can write some very quick logic that takes the actual barcode output and compares it to the information that we extracted and then raise our hand and say, “Hey, somebody needs to probably review this if the two of those don’t match.”

Jeff Cahill: Another question: How do you handle PII data? One of the big ones. Do you encrypt the data?

Todd Pratt: The application can encrypt the data in transit and at rest. Because we’re latching into some of your file storage systems in your databases, the encryption there would be sort of up to your IT staff. But certainly in our application and as we’re transmitting the data, PII or any data can be encrypted from a PII standpoint. We also have the ability to set thresholds. After a certain amount of time, the software can automatically clean itself up. The Hyperscience platform is transactional. We want to get the information in and then we want to pass it on to your line of business applications quickly. We also don’t wanna just sort of throw anything away in case there’s a transmission issue, you may want to resubmit it. So that’s really up to our clients how long they wanna hold the information inside of our application.

Jeff Cahill: Just adding a little flavor to the PII data conversation. Don’t forget what Todd had mentioned about our deployment options: we are an on-premise deployment option. So everything he talked about are your security controls built into your existing infrastructure. So if you harden your data, we are hardened. The other thing is to understand that we do deploy on AWS GovCloud as an on-premise solution. So all that is built into AWS GovCloud as far as controls can be built into Hyperscience. And this allowing you to get your ATO or your Authority to Operate actually quicker than you would if you were trying to do it on your own environment.

Richa Bharwani: Thank you so much, both Todd and Jeff. And that concludes our webinar for today, folks. Please note you will receive a copy of this webinar via email tomorrow. Thanks so much for attending and have a great day.