Driving Government Innovation with AI & IDP

Kyle Mckenzie: Welcome to Driving Government Innovation with AI and IDP. Today we’re gonna be talking a lot around how transformation functions within government, particularly around artificial intelligence and intelligent document processing. Today we’ve got two presenters. We’ve got myself, representing CharterTech where I head up our business transformation team, and we’ve also got Theo Popescu, who’s a senior manager of solutions engineering from Hyperscience.

Kyle Mckenzie: Today we’re gonna walk you through a number of things, including introducing hyper-automation and Hyperscience benefits to Australian government, what transformation actually physically looks like, and then we’ll jump into the system itself. Then we can go into a live expert Q&A. CharterTech delivers tailored and scalable end-to-end business solutions, particularly in transformation spaces. We operate nationally across Australia and we also operate within New Zealand. Part of what CharterTech and Hyperscience bring to the table is our IRAP-assessed platforms. We’re able to assist organizations utilizing highly secure platforms on Azure, AWS, or anything else that your organization prefers. We’re qualified ICT and accounting professionals, and we hold security clearances, particularly in the Australian domain. We are a Defense Industry Security Program certification, meaning that we can operate at the same level and host Australian government defense platforms, meaning we look after your data in the most appropriate way so you aren’t compromised when we’re working with your information. I’m gonna pass to my colleague here for a bit of an introduction around Hyperscience.

Theo Popescu: Thanks, Kyle. Hyperscience is a machine learning, AI intelligent document processing platform. The reason we’re here is that human-readable data makes up 80% of all documents, meaning that machines aren’t able to process any of those. We need to push them through some way of getting them into a digital format in order to run automation or integration. One traditional way has been using OCR. The plus side to that is speed and efficiency; they don’t take breaks and can run 24/7. But the cons are lacking contextual awareness. Looking at things character by character, the machine typically doesn’t have any of that context involved. It is often inaccurate. If we’ve got one character wrong, it’s not really 75% accuracy; it’s actually 0% accuracy because somebody will have to physically look at that one word.

Theo Popescu: It is usually difficult to set up and maintain. You’d have to create a layout for every single invoice supplier around, which is very tedious and expensive. Often there are limited use cases. Things like handwriting are out the window because we can’t support handwriting with traditional OCR, or poor quality where you need 300 DPI documents. When all of that fails, we fall over to the human task. That’s great because there’s contextual awareness and it is highly accurate, about 98%, but slow and expensive. Throwing more people at the problem only adds to the cost and complexity, hence why that lacks scale. You put all that together and feed that downstream, you have disjointed systems like RPA or CRM, and potentially you are providing erroneous data downstream with no guaranteed SLA of output. That introduces operational, reputational, and financial risk.

Theo Popescu: Hyperscience flips everything on its head because we are AI and machine learning. It is humans teaching the machine in order for the machine to learn—human-centered automation. We take the human-readable data and pass it through the engine that does the classification. Anything that can’t hit an accuracy target goes to a human. We have this notion of being able to provide an exact accuracy target that we want to achieve, which is typically 99 point something percent accuracy because that’s the level you need for downstream processing. Between the human and the machine, you should be able to achieve that.

Theo Popescu: We have identification, being able to identify fields on the page. Take the invoice example again. Instead of having to do multiple layouts, we just trained the machine to understand what invoices look like. Now the machine knows how to classify invoices and knows what fields are required and where to find them without us having to say “look in this position or this zone.” When it sees an invoice it’s never seen before, it can intelligently understand and find the fields. If the confidence is enough to hit our accuracy target, it will provide the answer. If it’s not, it’ll go to a human in the loop.

Theo Popescu: We’re hitting this accuracy target from classifying to identifying to transcribing or extracting the information. Out of the box, we have data types that have been trained on hundreds of millions of names, addresses, and dates in many different handwriting styles and formats. Anything that can’t do goes to a human in the loop. Over and above traditional IDP, we can do things in the platform that allow us to do validation of business rules. All of these things lead us to reducing average handling time and improving that whole process end-to-end so that what we see here is 99% accurate data flowing through to the end point in a machine-readable format.

Theo Popescu: Because we are machine learning, we can keep teaching the machine. That means the machine can keep getting better over time. It’s not what you get today is what you get tomorrow. There’s continuous improvement all the way through. That combined with our Flow Studio means that we can look at the whole use case and actually tailor all the specific validation rules and inputs. We can tie into an API in the middle or all the way throughout. Really complex use cases that we can work with, mix that in with the human in the loop for supervision and some great reporting, underpinning that whole thing with integrations. That makes it the mesh that’ll sit in between the whole environment. Whether we’re tying in where CharterTech comes in hand in glove with Hyperscience because of that complete ecosystem of operation, that hands off from RPA to internal systems. I’ll hand it back to Kyle to talk about some of the benefits of the Australian government.

Kyle Mckenzie: Thanks, Theo. How does that actually apply in practice, particularly in the spaces that a number of you on this webinar today would see? A core component around that is the functions that government particularly tend to offer. That ranges from federal government down to state government and local council, as well as any supporting industries like universities or non-for-profits. This covers some of the most document-intensive functions that exist, such as healthcare, education, law and order, defense and national security, immigration and border control, governance and public administration, as well as election management. What underpins a lot of that is applications and claims processing, FOI requests, invoicing, identity management (handling of PII information such as driver’s licenses, passports, medical information), and document storage in archives.

Kyle Mckenzie: A large part of what makes providing these functions to the public difficult is government must maintain onshore resources in order to provide these capabilities. This means that other cost-saving initiatives such as offshoring labor becomes impossible. CharterTech has built solutions here in Australia and in New Zealand that leverage security-assessed and appropriately managed systems such as Hyperscience and other RPA tools to provide these capabilities so we can reduce the cost of labor and effort associated with your teams to better enable them to focus on the tasks that they should be focusing on.

Kyle Mckenzie: Another core example specific to government is around the Freedom of Information Act. This has a large resource impost. Particularly when citizens are seeking information, whether that’s address, medical information, or information used in claims processing, it is up to the entities to provide that information back. This does not necessarily mean that all the information should be prevalent in the documentation when being handed back. This requires multiple teams worth of efforts to run and do. What Hyperscience’s capabilities are able to bring to the table is redaction. We’re able to take information and automatically take a document, classify it, and use field entity recognition to go “that’s a tax file number, that’s banking details, that’s an address,” and quite cleanly and accurately reduce that overhead in terms of redacting information as a first pass through.

Kyle Mckenzie: This limits the effort required to comply with regulation but provides better transparency, accountability, and public participation. It can prevent potential corruption in specific government organizations. This also allows you to run identity checks at scale with minimal risk utilizing the redaction capabilities and PII handling capabilities. It allows you to do retrospective processing. If you have a suite of documents that you know you need to have processed, you can have Hyperscience run through and strip that information out.

Kyle Mckenzie: One of the solutions that we have built as a firm utilizing capabilities such as Hyperscience is a fraud analytics engine paired with invoicing solutions. This helps do a risk threshold analysis for invoice processing at the transactional level rather than in bulk. That means you can prevent risk from occurring per invoice. We can check whether a vendor’s invoice is appropriate for processing each and every time before it hits your system. Transactional level verification of invoice banking details, liquidation notices, tax registration, phishing, and email language verification.

Kyle Mckenzie: The fraud analytics engine provides outputs coupled with the invoice itself for processing. For each invoice that’s submitted, it’ll have this report tied with it. You can see here that this specific invoice had a risk rating result of medium, and you can see the risks that were flagged in red. It’s looking at the actual ERP system, systems such as SAP or TechnologyOne or Oracle NetSuite, and checking the supplier’s information. Has the invoice issuer’s details matched at the address level? Has the appropriate email address been matched? Do the bank details match? Further to that, we can also at a transactional level check contract details such as the contract values, the contract PO numbers, and the cost of services increases.

Kyle Mckenzie: We also can look at vendor details, specifically whether there’s blacklisting elements, and whether there’s vendor and employee bank details associated to it. Additionally, we can then go out to business registration organizations such as the ABR and check whether the ABN or the ACN has actually been appropriately registered, whether the business name matches up with the invoice details. We can check whether they’re registered for GST. We can run ACCC checks for any infringement notices listed against that organization, as well as ASIC for any liquidation notices, as well as any email checks scanning for email language associated with phishing and a timestamp.

Kyle Mckenzie: Another core piece that we’re able to provide is around the ability to have an engine, which we call the intelligent processing engine, which leverages RPA for orchestration elements in order to take documentation from a specific portal, passing that through to Hyperscience for the physical document extraction for text, and then into a data analytics and data science platform for subsequent downstream processing. One such example is where Department of Defense utilized Hyperscience to support their works with the Unrecovered War Casualties for Army and CharterTech staff was supporting this development. This utilized National Archives of Australia information where there was 1916 information stored on the portal. This information was quite horrendous typically for staff to go through and strip out any key information such as medical history or service history.

Kyle Mckenzie: Our engine was capable of processing the full battle from Fromelles records for these soldiers at scale. This paired Hyperscience intelligent document processing to extract the data from the key documentation, RPA to orchestrate it, and then SAS Viya to create a catalog which allowed you to search keywords. It doesn’t matter if it’s soldier records; you could take this exact same engine and apply it in compliance spaces, wealth management, supplier records, or election information. You can apply this engine pretty much anywhere where you have documentation that you’re seeking to garner better insights to.

Kyle Mckenzie: What you saw there was the ability to take old information and produce data in a format that could then be utilized for overarching analytics. How many soldiers from this battle actually passed away due to gunshots? How many had medals? What was their service history? I’m gonna pass back to Theo to actually give you a bit of a live demonstration of how Hyperscience actually works.

Theo Popescu: Thanks again, Kyle. This is the platform. With Hyperscience, we’re trying to get the most accurate data in a machine-readable format. It’s all web-based essentially. It’s a one-pane view here to work with. Everything’s all in the one system. You don’t have to install separate verification stations with additional licensing associated with it at all. It is about that classification, identification, transcription flow where between the machine and the human in the loop. When the machine isn’t confident to hit our accuracy target, it will throw up its hand and ask for a human in the loop. Asynchronously, what we do is we have a QA task so we can validate and confirm consensus around the supervision that’s taken place in order for the machine to learn and get better.

Theo Popescu: When we look at the types of documents that Hyperscience can work with, we can work with everything from structured (application forms), semi-structured (invoices, payslips), and now totally unstructured documents as well. We can do sentiment analysis, named entity recognition. If you throw a contract at the machine, it will pull out every single instance of a name, address, and company name all built into the machine in one hit. We say bring all your documents.

Theo Popescu: I’ll upload a sample file so we can see how the machine operates in real life. I’m uploading this manually, but we support many different inputs: folder sweep, email ingestion, RPA, API, message queue. I’ve got the option of choosing from many different flows. In this instance, I’m putting it through a standard document processing flow: input, classify, identify, transcribe, and then output. First thing you’ll notice, it’s upside down, so we can see how auto-rotation works. But straight away, you’ll notice that we’re working with real-world handwriting. We’re not looking at printed letters in boxes. It’s whatever a human might be doing filling in a form. Things like outside the boxes, skewing of a really skewed document such as this. That intelligent visual page classifier will not only classify but de-skew as well.

Theo Popescu: Above the machine learning aspect, we have things like human intent recognition. So if somebody crosses this out like that, what’s the intent behind that? Other systems will try and recognize something. Hyperscience should hopefully understand that that’s not even to be recognized. We’ll see how it handles different date formats, blank pages, poor quality and handwriting. If a human can read it, then the machine should be able to read it as well. We’ll see how that works with photos of documents, maybe with half the page ripped off and things across it.

Theo Popescu: We’ll jump into the submission. First and foremost, we have classification. The documents have been classified correctly and the blank page has been left out immediately. What we’re seeing here is that two of these documents have one or more fields that the machine wasn’t confident to hit our accuracy target. Two of these documents are saying, “Hey, I don’t need anyone. I’m confident.” These, including the last document, are a hundred percent done by the machine.

Theo Popescu: If I click on performing tasks, this will take me into what the human in the loop activity looks like. First thing you notice, it’s zoomed in so that I can see what field it’s referring to. Hyperscience is field-level accuracy. We’re not talking about character-level accuracy. Either you get the whole field right or you don’t get it right. I can type in what I see. It’s made to be all keyboard driven to maximize speed. I type what I see, hit enter. That was the only field on that whole document. And you can see how already this has been de-skewed. Only one field that it wasn’t confident to hit our accuracy target. It moves on to the next field on maybe the same document, maybe a different document. To the data keyer, it makes no difference. They just get presented with a field to type what they see or hit escape if it’s illegible. We had two fields on those four documents that were recognized.

Theo Popescu: From a reporting standpoint, we not only have the ability to see what automation rate we had at our 99% accuracy target per document, but we can see two documents that I had one field that I supervised and the others that I had nothing supervised a hundred percent by the machine. That wraps up to a submission automation of 98%. You can start to really hone in and drill into the actual automation that you’re receiving from a system.

Theo Popescu: We’ll dive into one of the documents here. Thomas Edison, I’ve transcribed it. For audit purposes, everything is logged in. Straight away, you can see if you were running this through character recognition, you would be hard pressed. The machine looks at it as a human does, it has a confidence on what the word looks like. We can see how well it’s doing there with handwriting check boxes, mark true or false. Again, we’re looking at the whole address. Here is the data type that we put in. This small little thing here actually has the power of having hundreds of millions of addresses tied into it. It knows that it’s confident to say that these are numbers, these are words.

Theo Popescu: Not only that, but the machine will actually trace down any characters that move outside the box. Because Hyperscience has a thing called dropout, that original PDF that we uploaded, it’s as if you hold it to the light. It can actually take away and destruct the original PDF, removing all the fixed text that sits behind and only leaving the new information that’s there. In order to get the best, most accurate transcription possible. We can see stamps blurred. But because we’ve got this dropout, we’re just left with the numbers. And because we look in and around it, it will find the number. Not only does it find it, but we can do things like normalization, removing the dashes into just having the number remaining.

Theo Popescu: I will jump into the next document, which was our really skewed document that visual page classifier has done a fantastic job at de-skewing. Not only de-skewing but transcribing everything that’s on this page, including the human intent behind that crossing out. It has my details there, the date format. It’s recognizing one date data type that we are using here, can recognize any different types of handwriting styles. So when it sees this, this is the output that we wish. Any data finds in whatever format it may find, it will normalize down to a standardized output.

Theo Popescu: These last two documents which were done a hundred percent by the machine, you’d be my guess as to if you’re to look at these out of context, character by character, what they would actually say. And then you take into account the grain in the background. To me this is one of the most impressive demonstrations of what Hyperscience can do in order to give accurate output. Not only is it doing recognition of the whole words in this instance, it’s ignoring all this background noise. It’s also taking into account that the ‘Y’ drops below to the next box, but so is this a 4 6 1 here? It intelligently ignores it, knows that this is the ‘Y’ that belongs to the box above.

Theo Popescu: When we look at the document, the photo that we had on the desk, and I’ve uploaded the tax department’s including all four pages of information that has no reference to anything else. The only page that is filled in is the fifth page. It knows it’s actually classified the fifth page of that upload. We can see here again, I’ve got writing across the boxes, no issues with finding that. Even more impressive is how would you handle for something like maybe the ‘G’ going below the box? And then you’ve got the ‘P’ going above the box. So even if you were mapping this out without having the machine intelligently identify this, this would almost always go to a human to try and understand what’s going on here. But we’re recognizing it really well with the name data type.

Theo Popescu: I like showing this example here because we’ve actually mapped two things. This is character by character recognition. In this instance, it’s ‘C T O’. But when applied with email address data type, and again seen hundreds of millions of emails, it knows that in fact it’s more confident that the word is actually ‘crocodile’. So you can see the contextual awareness again at play here.

Theo Popescu: We spoke about totally unstructured. We looked at redaction. We have the ability to do things like tax file number redaction where we’re able to see a valid tax file number here. We have our output file which in this instance we’ve said we want to redact all valid tax file numbers. We’re actually validating this against the MOD 10 algorithm and the ATO. We can do this type of validation.

Theo Popescu: I’m glad you asked about flows. We spoke about how we did our standard IDP flow: input, classify, identify, transcribe, and then output. That output here is 99% accurate data that can then be fed somewhere else. What Hyperscience has the ability of doing is take that concept of flows and now build out a tailored flow. So if we are looking at a health claim for instance, we can look at the whole submission and say, “Hey, we’re actually looking for an application form, passport as ID, and an invoice.” We can classify the documents we want to have. The documents we don’t know about, we can run that through a full page transcription and run things like sentiment analysis.

Theo Popescu: In any case, what we have here is that same 99% accurate data to do something else with. And this is where the fun part begins because we can then say, “What do you normally do downstream?” We normally do a lookup and we say, “Is this a valid customer or user already?” Yes, they already exist in the system. Great, let’s pull their information down and maybe the name and address and let’s validate that. Did the name and address on the application marry up with what we have in our system? We can do inter-document validation. We can go out via API calls or work with RPA. So we actually build out a custom flow that ties in and measures in everything that makes sense to be done upstream. So when we reach the end point, we’re now saying, “Hey, we validated 12 out of the 15 points.” Now that’s much more complex information that we now have to deliver either to RPA or even back to a user to then be able to resubmit or provide additional information.

Theo Popescu: Hopefully this gives context to what we can do as Hyperscience from a platform perspective. It’s no longer just reading one character out of context like OCR has traditionally been. It is looking at the whole use case end-to-end in consonant with your surrounding RPA and other systems at play that makes the most benefit. And that’s really where Hyperscience and CharterTech work really well hand in glove in delivering these things out to government.

Theo Popescu: I think we had some questions that arrived. Kyle, if you wanna pick up that first question.

Kyle Mckenzie: When we’re looking at applying these technologies in broader contexts, going back to that intelligent processing engine I showed earlier, you can see how we can take unstructured information that is quite old, even dating back to the 1800s, and apply technology to extract that information and put into a structured element. Where there is no structure to the information coming through, CharterTech as a firm is able to push that through our text analytics engines that we built in Hyperscience to create that structure using natural language processing to generate those outputs that you’re looking for. It really depends on what you are looking to do with that information in terms of approach. But absolutely everything’s on the cards here. It is quite legitimately the most intelligent way that you could approach documentation and the extraction of that information out.

Theo Popescu: That’s great. We have another question: Have we been able to solve the issue of reading cursive writing? We see this on historic documents from the 1800s, certificate of title surveys reports, etc. Kyle touched upon that as well. The Fromelles battle forms, that was a lot of cursive writing that was in that, and we’ve done things around birth, death and marriages all the way back from the 1930s and the like. So these are use cases that we’re familiar with and we’ve had pretty good success with capturing and extracting data from there.

Theo Popescu: The next question we have here: What security assessments does the platform have?

Kyle Mckenzie: We’re going through the IRAP assessment process as we speak. So that means that the platform itself will be ready for Australian federal government. It’s gone through FedRAMP in the US as well. And it also has a whole bunch of the other ISO certificates that will be required in these spaces. Most of the certifications are in train, looking to be completed within the year or so.

Theo Popescu: Another question: What is the acceptable error rate for the bot? Is there a quality verification process? With Hyperscience, we look at training. As I mentioned, we have a QA process there. So we have consensus. Effectively when we’re training the machine, there is that consensus value, which means the machine and a human must have consensus. If not, then a human and a human must. So we will arrive at a consensus of what the actual word was. That provides the machine the ability to have that confidence on delivering that target output or hitting that target accuracy. Now we need to trust in the machine that if it’s not confident to hit that target accuracy, it won’t give you that answer. If you’re setting the accuracy target lower, say 98%, 97%, the machine may then offer you up its answer to hit that. So it may have three out of a hundred times it might get it wrong, but typically that’s the way that it operates.

Theo Popescu: Does running this type of AI require many resources in terms of hardware or cloud computing power? From a Hyperscience perspective, not necessarily. All we need is typically an application server, a trainer, and a database server that we connect to. And that gets geared up for how many documents you wanna process typically within a 24-hour SLA. But minimum specs are eight cores and 32 gig of RAM is what’s required out of the application server to run. But then we scale it for speed there.

Theo Popescu: Next question we have is probably around pricing. How does pricing work? Well, typically it’s per page. So within the Hyperscience pricing, obviously you may have professional services in or around the work that gets done, but it’s all cost per page. So you can have as many users as you like on there. And it’s all at a price per page. I think we’re coming up to time, so we might leave it there. Thank you very much everyone for attending.