Insurance Claims Are AI's Messiest Data Problem — Denys Linkov, Wisedocs Transcript, AI Summary & Key Points

00:12 All right, thank you for joining uh the last workshop of the day. So, I'm Dennis. I'll be sharing a little bit about uh insurance claims data and how powerful it can be for enterprisewide AI programs. So, let's get started. So, who here works in insurance? Few people who here wants to work in insurance. Okay. Interesting. Uh a few people. All right.

00:35 Um so insurance is a very interesting uh sort of sector of our economy where people sort of find their way working there, but it powers such a important part of our ecosystem. So we're going to talk a little bit about uh insurance as a whole. And we're ultimately going to focus this talk on how do we improve our lives in general. So we'll focus on medical claims.

00:55 will improve the claimment and patient outcomes uh as a focus of this talk and I'll keep bringing that back. I think oftentimes in a lot of fintech uh sort of finance insurance discussions sometimes things can get a little bit too business focused. So I always want everybody to to focus on this aspect. So a little bit about myself. Uh I'm Dennis. I lead the AI program at Wisdocs.

01:18 Uh we help companies with their complex medical claim processing. I've been with the role uh in in the role for two years now and we've seen a huge amount of transformation since 2024. So you might have seen some of the news coverage in terms of medical coverage and insurance in general. Um the whole GLP1 uh transformation in our economy in our healthcare environment has changed how insurance is pricing things as well.

01:44 uh we also see that spousal insurance and broadly insurance coverage has continued to evolve with increasing costs uh and other factors that change the the approach of insurance. So the talk is going to focus on three main chapters. The first one is on the technology transformation that we have. The second is I'll go a little bit into the insurance space uh in terms of some of the call nitty-gritty but sort of a brief introduction there.

02:07 And then also I'll talk about how AI is going to help make insurance and the patient experience even better. Let's talk about technology transformation. So we all probably in our work have started some form of digital transformation within our companies whether we work in smaller startups or with in larger enterprises. Now the challenge with that is that we never finished this transformation.

02:29 AI hit us in 2023 and 2024, but most companies have not finished their cloud migrations, have not finished leveraging big data to its full potential, have not deployed a clean data lake, right? Who here has a data lake they're proud of? All right, very very few people. Is it even possible to have a data lake you're proud of? Right? Uh and on top of that, we're now getting all this hype about AI and we need to really make sure that we have the other prerequisites down before we can actually deploy real applications.

03:01 So, in insurance and many other industries, uh, paper was the big, uh, way of processing things, right? You'd get a stack of papers of going through a medical claim. You'd organize it. You'd have these giant tables full of them. You'd use cards to put in certain sections. And this is very difficult to actually get any anything outside of the claim itself because how do you do crossclaim analysis when everything is paper, right?

03:24 You could put as many folders as you want, but it's very difficult. Now, we still have not exited the paper era. There are still a number of practices that that use paper uh around the world, whether it's in the doctor's office, in in legal departments, or within insurance companies themselves. So, we are still not over paper, but we're getting close.

03:43 Now, in terms of digitizing, uh we are still also not over the fax machine. The the fax market is continuing to grow. Uh it's over seven billion dollars globally, which is always surprising. Uh I referenced this uh in my talk at AI engineer SF last year and it has not changed. Fax is still being used. Now what we're doing now is we're jerryrigging faxes into our digital systems where we can convert them in a fairly clean format.

04:06 Uh but it still does not solve the problem of how we actually exchange information in the healthare system. Now if we think about how quickly technology has been adopted uh EHR's uh electronic health records uh were only at 8% adoption in 2008 and only with the the COVID pandemic did we actually hit the 80s and 90s uh in EHR adoption and again many hospitals still run on paper.

04:33 So we are still not fully through this digitization digital transformation era and yet everybody wants to do AI. Now, if you think about the types of forms people um receive uh when when doing claims processing, uh these are some three sample forms from three large states uh for workers compensation. So, workers compensation is if you get hurt at work, you file a claim saying, "Hey, I hurt my arm falling down or I was driving a forklift and then something went wrong."

05:00 Uh and none of these forms look the same, right? So, within 50 different states in the US, you have 50 different types of forms. Generally, they all ask for different information. and they're not standardized. Uh they're fairly standardized in that every form that comes in looks the same. So you can have some determinism there, but even then the information is different.

05:18 So even as a country in the US, let alone any other country in the world, we don't capture information in the same way. And these versions sometimes change as well. Now if we talk about what people care about in the insurance space is a loss ratio, right? So how many premiums are you collecting and how much money are you paying out? So I'll talk a little bit more about uh the these numbers here and what they mean.

05:44 But in general, this is what drives the bottom line of an organization, right? When you report a quarterly earnings or an annual earnings, you're going to be emphasizing what is your loss ratio, how well can you predict risk, how well can you underwrite, how how well are you managing fraud, how well are you managing your expenses. And so companies want to keep costs down and make more money, right?

06:03 That's just capitalism in general. Um and if we think about some of the statistics here, we we generally uh hover around the 50% um loss ratio here in terms of the amount of money we have to pay out for insurable claims uh compared to the premiums we get. Now I'm introducing all of these sort of pieces of context and I've only said AI once so far in my talk and it's really important again to think about the problem that we're trying to solve and then only choose AI if it's the right solution.

06:37 Now for AI uh to get proper AI solutions we need to go through the right amount of transformation right we need to have the right goals in mind we need to have a team that can actually articulate what we want to do with AI we can't just have people get on stage and say AI AI AI right we need the right people skills to actually accomplish this transformation and we need the right frameworks in place whether it's governance security infrastructure management and data now as a question here um for those working at an

07:03 enterprise context text. Who here could get a database or a VM within a week if they requested it? Okay, who could not? Okay, that's progress. Who here could deploy an application to production in a week? A new application? Okay, who could not? Okay, so so we we still need some progress. Who can choose a new AI vendor in a week? There we go. No hands.

07:26 Right. So it still takes time for us to go through all these different processes. Uh they're certainly better than they were 10 20 years ago in the digital transformation component, but a lot of companies still get stuck in governance and security. And a lot of those have to do with with the factors here. Who here has been asked for some kind of query like this from an executive?

07:51 Give me some data set that is perfect data, right? A few people, right? We we we've seen this before. Hey, can you pull that report for me quickly? um it's much easier now with AI but you need to assume that that data exists right so if we come up with a more complex query here right uh let's say that we we have a medical claim and for a given patient and their condition uh in this case uh an ankle injury we want to make sure that we can assign them to a doctor who has top 50 percentile uh patient outcomes right within

08:22 some recent time frame now this query is fairly complex, but if you could actually just write this query on your data set and get that result, right, I'd be amazed, right? Data is very very challenging in an enterprise environment. So for us to get these kind of outcomes and get these kinds of insights, we have to structure our data systems in in a in a very well- definfined way.

08:49 So let's talk a little bit about about medical claims. So in general we've grouped uh insurance based on the the NIC uh classifications. We have property and casualty, life accident uh and health and sort of broader health uh insurance. I'll actually be talking about property and casualty since there the claims are typically the the most complex. If we think about PNC um I mainly worked on the casualty side.

09:13 So uh auto uh insurance uh various types of liability, general liability, product liability, workers compensation. There's also the whole property side uh which is its own uh set of complexities but generally this is a $1 trillion market right uh based on on premiums and reinsurance is its own can of worms. So if we think about these blue boxes many of these handle complex medical claims somebody gets hurt because a product is defective somebody gets into a car accident uh somebody gets hurt at work.

09:47 Now, adjudicating a claim and getting a response back to a patient or a claimant takes a long time. In the best case scenario takes a few weeks. In the worst case scenario, it can drag on for a few years. Depending how complex the the scenario is, depending how complex the regulatory environment is, depending how litigated the environment is. And even once you get the the medical services delivered, the claim can keep dragging on and on.

10:09 So, this is not a simple process. And I want all of us to imagine how much more effective the the healthare system could be if we could shrink this by a factor of 10. Right? You get hurt at work, you you submit the application, uh you go through that process and everything is digital and everything is smooth. So why does it take so long? There's a lot of handoff points with getting even the medical side, not accounting for for legal side, the actual insurance side as well, right?

10:39 So when you when you get hurt uh in one of these scenarios and have to be insured for a complex claim, you have to get the the diagnosis. You have to go to various specialists. Sometimes specialists disagree, right? Then you have to make sure that you have the right coverage. Uh whether it's primary care or one of these uh additional forms of liability.

10:58 You have utilization reviews where doctors have to agree with one another that a procedure is necessary. Uh we have treatments, we have billing, you try to figure out what your bill means, right? with the explanation of benefits and coverage. You get the payments and you have continued treatments. Now, there's so many additional bubbles here that can be added and forks in the road um depending on on the steps, but in general, it it is not a simple process and along the way lots of things can go wrong.

11:26 Uh I just listed around 15 that that come to mind immediately. But this has to do with the HR situations with insurance companies with claim adjusters having high turnover, higher retirement rates. Has to do with technology of people reprocessing the same data over and over again. Has to do with the types of uh situations we end up with where claimants are are stressed and sometimes make mistakes uh and are in the position where uh they're not acting rationally.

11:54 Right? because if you're you've gone through a traumatic car accident, you're not going to be the most rational person. So, let's talk about where technology can help us. So, let's go through uh a worker's compensation claim. Has anybody seen a workers's comp claim before? Few people. Okay. So, typically when somebody gets injured, they have to report uh it within the workplace and then make a filing of that as well.

12:18 uh those three forms I show, the Texas, New York, and California ones, those are typical intake forms. And you go through this process of actually visiting various people, whether it's in the emergency room, whether you have additional treatments, and so forth. And you don't always have your claim reviewed until you get claims later on. Now, as you go through that, there's various types of claims.

12:41 Here I generated some synthetic ones as an example. U but they're they're fairly close to what you have in in production. So you go through you go through all these types of treatments you get different types of procedures. You might be in network, you might be out of network, you might be traveling for these and sometimes you have conflicting information as well.

13:03 And as we go through each of these processes, all the documents look different. Every hospital has its own way of of uh structuring a bill, right? Every doctor has a different way to write a progress note. It it's not consistent in any way, shape or form. So, this is why the data is unstructured and many times it's not in an electronic feed. It's a PDF.

13:22 It's an email. It's a fax. So, you have all these technology complexities that layer on top of all the business complexity that lies underneath. Then you have something like this, right? somebody does an assessment and there's three different results right for let's say uh the range of motion that somebody has how are you going to resolve this uh in order to adjudicate the claim now we haven't even layered the the other portion on top of this while these medical processes are going through there's also the risk of

13:54 litigation so within insurance uh typically insurance is on the the defendant side of a litigated claim and you have the plaintiff side now the plaintiff side is incentivized to handle as much volume as possible uh and to get as big of a settlement as possible. So you actually get into a situation where you have two different incentive structures. So on the pliff side goes quickly as possible, make the most uh the biggest claims possible, put up giant billboards everywhere.

14:22 We we saw those when we were flying in probably, right? Uh all over the place. uh and the plaintiff side has actually adopted AI a lot more quickly compared to the insurance or defendant side because you get a percentage of how fast you move and how big your claim is. Now the range of the actual percentage of compensation for for lawyers depending on your jurisdiction can range from 9 to 15% if you're in California for workers comp uh fees in other jurisdictions you get into that one-third territory or even 40%.

14:49 So now on top of healthcare, on top of insurance, on top of technology challenges, on top of administration, you now have litigation and insurance companies set aside a lot of money for for litigated claims. Now the challenge here is that if you have not gone through a proper digital transformation, you are especially vulnerable to to bad actors, right?

15:13 So you can be sent repeated claims, you can be sent fraudulent claims is making that even easier to do right now. So if you don't have the ability to actually handle those and process things efficiently, you get more and more fraud in the system, right? It's similar to if your website is easy to do, right? People are going to keeps you and we forget about what matters most as we start solving all these sub problems.

15:40 Right? So these are the things that matter most in terms of the quality of somebody's care, reducing costs, making sure it's a quick process, and it's a good experience. So let's go through and and solve some of these problems. I'll talk about three different phases and how they can help. So number one, let's talk about clean data, how we can accomplish that.

15:57 Number two, let's talk about within document analysis. And then let's talk about cross claim factors. Okay, clean data. So has anybody here worked with claims data before or any kind of insurance data? A few folks. Okay. How many duplicate records sort of have have you processed or have you seen? Many. Yeah, millions. Right. Uh depending on the jurisdiction that you work with, they can be as much as 50% or even higher, right?

16:28 So you have so many duplicates coming in and sometimes this is intentional, right? Sometimes to get a claims processed, you actually have to keep pinging people, right? Following up with my claim, following up with my claim, right? Uh so because people are slow you get into this sort of cascading failure effect that we see in technology co-mingled records with other patients right you have Bob Joe and you have Bob Joe Jr.

16:50 Right? Those medical records get mixed in at a at a fairly often occurrence. It violates HIPPA. There's a whole bunch of regulatory risk for that, but the system and the digital transformation is not set up for that. And decisions can actually be made when other people's medical records are are mixed in. Right? So, this is especially sensitive when there is a high uh complexity claim where things can be missed.

17:14 Somebody's record gets mixed in by a doctor upstream or administrator and suddenly the whole claim falls apart. Then we have the the more expected items of poorly scanned or undigitized data. Right? You have some some handwriting that you can't understand, let alone doctor's handwriting, right? Some things come in on paper. So we we keep getting these poorly uh clean data examples and they happen for many reasons, right?

17:40 So some of this happens because uh you have a lot of copy pasting that happens in the medical system, right? you go through uh doctors or nurses as as they're updating the medical records, right? They'll copy paste between patients, they'll go through and they'll copy paste between visits. Uh you might have seen that personally, but it happens a lot uh in the space.

17:59 Uh EMRs are actually not set up consistently. So, has anybody here seen an EMR before? Their own medical record, right? Anybody here moved across providers or networks? Yeah. Do they look the same? No, they don't look the same. Um I'm from Toronto, Canada, and many of our hospitals do fresh implementations every time they set up an EMR. And this often happens both due to the initial processes that exist within a hospital.

18:27 It happens across a network as well. There's very interesting dynamics that happen on why it's set up a certain way. And if you want to talk about data cleanliness, people do some very strange stuff in EMRs with nested fields. So even though we do have electronic health records and medical systems, they're all different. They're structured weirdly. So you do have to do some some transformations on it as well.

18:51 And often times it's it's difficult to pull data out. So people just choose to to send faxes or PDFs at the end of the day. So solving clean data is not an easy problem. Uh AI can help us, digital transformation can help us, but upstream there's a lot of factors that go into it. So uh this is an example of some illustrative data uh of duplicate records.

19:11 Uh real data looks quite similar as well with a high number of duplicates across certain networks. And this is actually very frustrating if you are on the receiving end of these duplicate records right so we have a number of customers who are like can this organization stop sending us so many duplicates right and unless you have good systems to detect this you you just keep getting the same bad quality data.

19:32 So it's really important to have those technical implementations and detection systems to actually make sure that you can safeguard your your own data uh data um cleanliness. Now once you have this data there there's mechanisms of of actually fixing things right for regulatory or uh interpersonal reasons right but you still have this messy data that exists as a baseline and once you can detect things like duplications uh again seems like a solved problem uh you no longer incentivize things like sending claims multiple

20:05 times with with different dates or the same claim uh or just other folks have to clean up their data as well. The other part that beyond just the cleanliness of the data is that these claims are are really large. Uh when I first started working and saw within our database that there was a PDF that was 8 gigabytes in size, that was very surprising. Um so these claims have anything and everything within them.

20:32 They have uh images, right? Uh if you think about auto accidents, right? There there are images uh within the site of the injury collisions and such. There's X-ray reports, uh, ultrasounds, everything in between. And sometimes this just crashes a lot of PDF viewers. Uh, if you think about some of the old computers that you might have, trying to open up an 8 gig, uh, byte PDF will be will be quite challenging.

20:54 Uh, and even on the technology systems, right, we talk about context windows all the time. How are you going to fit a context uh into a context window an 8 gigabyte PDF, right? It's it's not as easy as saying, "Hey, Claude, take a look uh at my 10-page PDF and and give me a response." or hey chat GPT or Gemini right it's not that easy um and on top of that actually going through and working through a case requires a significant amount of time so these are the technical challenges uh or the business challenges that

21:20 result in very complex technical challenges now if we work through and we're adding more and more complexity into here a lot of these claims also have irregularities so you might have some progress notes for example of a earlier patient report that says hey here's this x-ray that was booked for this date, you receive the claimment package and that X-ray is not there, right?

21:42 And it's very easy to miss that without actually having this highlighted in some sort of way. Um, and without that again, the whole trajectory of a case can change whether there was adequate care given uh whether the person actually went to the the care provided uh and how do you actually cross reference this with the billing side of things. So this is a very very common scenario where upstream again because the data is messy you actually have to ask repeated times hey can you send me this record oh we missed faxing

22:10 this oh the fax glitched oh we changed our backend system right so with as an organization you have to have these lines of defense in order to make sure that these get flagged and as a claims adjuster claims adjuster might be processing 200 claims at a time so they're not going to be able to keep everything within their mind saying like oh yeah I need to follow up on on this X-ray or not, right?

22:31 They can take notes, they can move forward, but things are going to get lost. So, this is again where technology can be a great lift uh to these kinds of problems. Then you get conflicting diagnosis. This happens fairly often as well. So, who here has sought a second opinion for a medical condition? Few folks, right? Um sometimes we we seek them out um to get another perspective.

22:56 Sometimes you get completely different answers, right? And the more complex a claim is, the more likely there is to be some some kind of conflict. So in this case, uh what often happens is when you adjudicate a medical claim and you see some conflicting uh diagnosis or you want to doublech checkck how much you should set aside to actually pay out for the claim, you send it to an IME or an independent independent medical evaluation.

23:20 Then you have another doctor who performs their their review of the file. And then you have to manage those doctors and the provider networks and the IMEs themselves. So now we're stacking again more people into the ecosystem, more complexity. Now AI can be quite useful here. Um and it it would be I guess poor taste to give a talk in 2026 without talking about evals.

23:43 Um but we did use AI to to try to solve these issues uh in some ways the conflicting diagnosis, the missing records and a number of other ones. So we built this benchmark called MLCR uh with artificial analysis to to measure the effectiveness of AI and actually answering these questions. So flip over to our blog post uh that we talked about and I'll scroll down to what this benchmark looks like.

24:08 So we released uh six different tiers of questions related to medical claims. Some of these are very simple like when did this injury happen and what was the injury, right? Then they get a little bit more complex. How many surgeries get performed? Uh how much capacity does the this patient have? And eventually we get into more and more complex questions that are even harder to to resolve.

24:30 And oftent times the way that you ask a question as a claim adjuster, a doctor, an administrator, you don't have full context of what's happening. So this is where we see hallucinations show up where people assume something is in a medical record, but it's not. So let's go a little bit more into the results. So uh this is available on the artificial analysis website um if you want to take a look.

24:56 Uh so we released an open-source uh data set for the first three tiers of questions that we have the easy ones and we have a private set that we host with artificial analysis for um to stop benchmaxing to to a certain extent. So it's very interesting to see how how models perform. Uh we actually measure this on three different uh axes. So let me flip back to that slide.

25:18 We measure this on conciseness. So how concise is a response? Uh and we measure this against a human uh expert validated answer. We then go through accuracy is that is the answer correct? Uh and the final one is completeness. Are all the relevant uh informations uh piece of information answered uh in the question. So if we go back here um the frontier has continued to evolve very very quickly and this is where having eval is really important.

25:45 uh when we released the EVEL uh the highest accuracy that that we saw on the levels of reasoning that that we had was 20%. A few months later we're in the 75% with with the very high reasoning levels, right? So models move very quickly. Um but this is great because it means that we can start productizing a number of the more complex questions and we have more certainty of the confidence that somebody has to review this uh in a system.

26:11 So within our system there's always a human in the loop. Uh so that's really important. You also start getting the the cost curves here on the predto frontiers on the cost to actually ask this question and get a suitable answer and this has been improving through the past few months as well and you get some of these trade-offs as well. So one of the interesting things when we were running these benchmarks on uh document analysis was that the GPT models were were excellent at accuracy but they emitted some key pieces of

26:39 information. Uh anthropics models were were quite good. Um and they're much more complete on the amount of information. Um they they do have slightly lower accuracy here as well. >> Can I ask a question? >> Sure. >> Accuracy. How do you measure it compared to the human factor? >> Yes. Compared to to human answer uh and then we do the triple judge uh verification.

26:59 So yeah. So we have accuracy here. You can see how the curve looks like and we have completeness uh which is uh is still not solved across most model providers. And yes to to the earlier question, uh this is based on a human verified answer and on a calibrated set of judges. Uh within our blog post, we describe the the methods we have here. Uh and we released a follow-up as well on on how we did this calibration.

27:27 So even though we're using AI here to structure our data and extract important insights, we want to make sure that each step uh of that process is well calibrated because if upstream uh your data is incorrect and you start using it downstream, right, you haven't gained anything. So again, really important as you're going through the claims uh uh process to to calibrate your judges to build good evalu at most talks.

27:54 And you want to make sure you have those experts who can actually verify this information. So you work across teams to to get there. Okay. So that's within a claim. Uh we know that with some amount of certainty we can extract some types of insights. We can flag conflicting diagnosis. We can dduplicate. We can search for missing records. Let's go to something more interesting which is cross claims.

28:19 Right? So, if you are a midsize uh insurance company, you might be collecting $1 billion in premiums within a uh portfolio of lines of businesses and you might be paying out, let's say, 20,000 uh claims uh at a time per year uh or even more depending on what line of business you deal with. Now, you want more insight of where you actually like what's affecting your loss ratio.

28:41 What kind of claims are people making in which geographies? What type of injuries? and what type of doctors are being involved. So this is where I think it gets quite interesting as well is actually being able to once you have that data extracted from the claim, you can start doing interesting visualizations, regressions and work as well, right? So two of the questions that we we've been asked and we've uh worked on answering is sort of how much are people traveling within a network, right?

29:09 Uh so if a patient is let's say in San Francisco or New York, right? are they getting treatment in their local area and is that the right person to to be giving the treatment. We also have these networks of who's actually working together. So this is some redacted data uh a subset from from California and you can actually track to say which lawyers are working together, which doctors are working together, uh which chiropractors, physiootherapists and what is the actual uh working relationship look like and the story

29:37 within the claim itself. So this is that sort of higher tier insight that you can get within uh extracting data from your claim. You then get into some interesting medical outcomes, right? So which conditions co-occur together. So if somebody has a back injury as an example, right? You might have a series of different diagnoses uh that could be mapped to ICD codes that coexist together.

30:00 uh and you might see okay why does this exist from a certain uh doctor's uh write up versus another and in this case again you you only get these capabilities if you can solve the data extraction problem within one claim and then roll it up into your broader uh insurance portfolio. Now we get to the money portion, right? So pulling a small sub- segment here of two treatments uh of physiootherapy.

30:30 Different doctors charge different amounts, right? We've probably seen that. Um and sometimes that's fine. If you're a better doctor, you should be able to charge more, right? If you can get somebody to recover as quickly as possible in five treatments instead of 10, you should be able to charge more, right? So uh but it's really important to understand costs and focus on minimizing them because as as soon as there is cost inflation through different parts of the medical system it keeps compounding right premiums

30:54 increase uh the fees that we are charged increases uh and we get less for our money. So within this analysis uh you can see across claims is there a consistent factor what what is the amount of coverage that you should be providing? How how should you be underwriting these costs for claims as well? And you can estimate how things are going to change over time, right?

31:12 So year-over-year, as an example here, uh there was a 10% increase for this type of treatment for a given doctor, right? So is that going to happen across uh the jurisdiction? Is that going to happen across the the industry in general? So then you can start doing some counterfactual analysis or or changes in in analysis to say if we change our network, if we start directing folks into a certain direction, if we change how we how we bill, right, what are the potential savings that exist and what kind of procedures

31:45 should people be having based on the standard of care that exists? Um there's there's recently an article about how has anybody se seen at the over billing article uh about $1 billion of additional billings that have come in? Anybody seen that? Um so those are examples where again if you cannot process data effectively people are going to find ways to to to go through and and submit additional records uh or do some some shadier behavior as well.

32:12 So all this to say is that once you have good data you can start making these decisions. And then once you have this decision uh this claims information you can start writing it into a broader system right so you can start using this in in other lines of business of saying if you know these are the trends in auto injury or workers compensation are we seeing the same thing in general liability uh if you know that your underwriting model is missing c certain trends uh or you don't have as much explanability there can we

32:45 make the model more explainable with what we're seeing um how do we provide this sort of magical data table that we wish that that existed uh within our data link. So again the more data you have the more complete of a picture you you can make and the better you can underwrite and price the product at the end of the day for the consumer and this is the information you need to actually power the the effective AI applications whether they're on on the underwriting side uh on the fraud side uh on the claim side as well.

33:16 So if you think about like a blue sky world, right, we we want to reduce something more broadly. So if we want to reduce costs for everybody, right, taking into account that patient experience, if we want to reduce the time that a claim is open, if we want to improve how quickly people get back to work and the the treatment they get, we need to make sure we have the right data and the right digital transformation to do that.

33:38 So this is an illustrative example. uh in the work that uh we've done within wise docs as well we've seen significant savings as well um so going through this digital transformation process and having the right data is really really important so I'll wrap up the workshop here I did want to leave quite a bit of time for questions because it is pretty comprehensive topic uh so thank you for listening and happy to take some questions blog post >> yes I'll I'll share the blog Yes.

34:11 Question over here. Yeah. So, so the question was about how do we actually structure the data to build um all this information. So, uh it goes through a number of different levels. Um I'll share some talks that I've given on on the topic as well. The baseline is we we use OCR and images to to extract information from the underlying records. Sort of standard pre-processing.

34:30 Uh pre-processing PDFs is not an easy task as well. if anybody has worked with them. Uh those 8 gigab byte ones have some some weird one in a million examples in them. So once you have the the data, the text and images, we have a series of models on top um NLP models, extraction models. We use VLMs uh to structure it. Sometimes like a more standard entity extraction model works well.

34:54 We have various classifiers that we build on top. Um we then have a summary layer that sits on top of that to summarize the documents. uh that is then human verified to make sure that the information is correct and finally we run those additional layers of analysis on top of that. So we have this abstraction pattern that sort of concentrates the the density of information as you go up um but it's verified at various points in time with various models and and human intervention as well.

35:18 So that means once you get to the top and the highest value items that's when AI can be effective and those are sort of those MLCR uh type questions that can have the right answer. So within MLCR this actually uses clean data to ask these questions. Real world data is not clean. So you have to have various steps ahead of that to to clean up the data to get there.

35:39 Yeah. So typically our our customers so bigger customers have multiple lines of business internally and they have different uh rules I would say across sharing across different teams. Uh other customers just work in one line of business. So they might just work in auto, they might just work in disability, they might just work in workers comp. Um but we take the trends uh across the board to try to figure out and flag those.

36:00 So there's the question that within sort of an office right how do you surface some of those insights uh or or sort of geography then you have the the layer on top which might be um the broader company but comp insurance companies also have a collection of companies within them sometimes uh separate businesses uh and then how do you actually feed that into a centralized data lake uh as well.

36:21 So again, every company is a little bit different, but we we go as fast as our partners want us to. So we live in claims. So So we focus on claims. So claims are are our bread and butter, specifically complex claims. That's where our whole AI pipeline uh is focused on. uh but as we sort of evolve and go and solve the sort of easier challenges I would say with adjudicating a claim, processing a claim, extracting that information, you start seeing these more sort of insightful aspects that help uh organizations make

36:52 portfolio level decisions. >> Yeah. Yeah. So that's a good question. So the question is how do we on board somebody when there there's a lot of these challenges? So typically we work with the the most important thing is a willingness to change, right? So we we've all probably worked with teams or organizations who want to change versus those who don't.

37:10 Um very very different experience. So assuming it's an organization that wants to change, uh there'll be a few sample files typically that we'll work through uh in various formats. Typically people will send us either things they're immediately working on or have or basically are trying to back test. Uh and then we go through a review process to see what did you extract?

37:29 Uh what did our system extract? How does it align? Does it match the information you want to see? Do you have different types of uh standards? So if we go back for example uh to the different types of documents here, let's just go back to the intake form. There might be various dates on this form. Do you want the the date of injury or do you want the the report date when it was submitted?

37:51 Right? So these are things of of alignment of business objectives. Go through process a few standard cases. If things work well then we start sort of a broader pilot and then gets into the fun integration aspect of integrating with with upstream providers apart from just like hand uploaded documents API integrations like dealing with all all the messiness.

38:10 So as long as there's a a clear business outcome and transformation goal then it's it's a fairly straightforward process. Um and again as as we go through that journey again sometimes it's like we don't have the capabilities out of the box sometimes we do sometimes there's multiple integrations on internal systems as well. So >> yeah, so let's start with the first part of the question which is handling regulations and how do we work with partners.

38:33 So we it depends how how quickly people are are moving. Uh it also depends how quickly regulations are moving. So the the way we've set up the system right now is that it always requires a human in the loop to make a decision. Uh so we surface these insights as an example for things like conflicting diagnosis. We'll see hey on page two on page 20 there there's a difference of diagnosis between these two doctors.

38:56 you should take a look. Right? So, as long as there's a human in the loop, I would say it it removes like 70% of the the regulatory concerns that people have. Uh regulation is also still catching up. So, uh there's a lot of things to read about like the NIC reports that I mentioned. They that organization publishes a number of different reports. There's the guidelines that exist for that.

39:17 We talk to the internal governments and regulatory teams as well. uh and if they want to see some pieces of information like we will share with them like this is how the AI pipeline works right this is the type of data that's being used um but at the end of the day as long as you don't build a black box and things are auditable uh and there's a human in the loop that take cares uh takes care of many things the regulatory side there's like a whole other like business decision decision aspect like how to approach nuclear

39:43 verdicts uh for uh nuclear verdicts are um very large uh awards or um claims that that are $10 million or thermonuclear ones which are $und00 million which really reallocate uh businesses reserves. So for those ones typically people come with like an approach on how they want to handle them and within the platform you can say I'm going to handle my most complex claims first or I'm going to handle the one that was submitted first.

40:09 Um and those are all sort of business decisions people can make. Uh the second part of the question on like people's internal like organizations internal goals. Um I mean we want to work with partners that share the same sort of mindset about improving patient care uh reduce same backlogs uh for claims providing the best decisions. A lot of times our customers are not even thinking too much about like the the cost of healthcare itself is everything around it.

40:36 Right? So, uh, if you're a large insurance company, you actually spend hundreds of millions of dollars on legal fees, right? So, that is something that is not even tied to the patient experience that there's a large amount of optimization you can do there. And to speed things up, right? Um, there's regulatory concerns that in certain jurisdictions like California, things are very litigated.

40:57 Um, you want to figure out how how to deal with those. Um, there's a whole bunch of middle parties that that are involved as well, uh, that are also inefficient. So depending on the customer, everybody has a different goal. Um, but in our conversations, it's it's always been like we have this very obvious problem to solve with technology. Let's solve that first.

41:14 Um, and again, the insights start surfacing afterwards of being like, oh, I didn't know that about my business. Let's solve that afterwards. So yeah, so the question is about decision APIs in Jev. Um, as part of their I don't know if anybody's seen the paper um copies of the AI news that are that the conference is handing out. There's actually an article that I'm writing for that.

41:35 Um, so we we've been using classification models uh for a long time. I've generally had a strong opinion on that. Hasn't come out today. It's coming out in a later edition. Um, I think it's very useful. Um, but LLMs are not the right solution for everything, right? So when we talked about this, right, this segment of AI can be split up as well, right?

41:56 Classification, extractive U versus sort of the more complex decision- based ones. Uh, so for us, cost is really important. So that's why we trained our own models as well. Accuracy is really important and hallucinations as well. So I think Jev was not a surprise that it came out. It was more so a surprise of how long it took because I think the LLM like token max reasoning for every problem uh phenomenon lasted for a very long time until we started thinking about costs a lot more this year.

42:22 So uh once uh Jav is sort of hippo already we can test it out. Um but we've sort of built our own models with the same paradigm in mind. So yeah. So I'll answer part two. Um so as an organization we have not moved the broader industry um to the extent we want. Right? So to say that we can say we've reduced the the claim uh time by 5 days on average for somebody across every patient.

42:48 We'd love to say that we we're not in that position yet. For specific organizations like we had some case studies on the website. I quickly put put some up as well. There's some significant ones within lines of business uh for cost reduction and speed. Um, so you can check those out and we can chat afterwards as well if you're looking for more specific sort of context on that too.

43:08 Yeah. So, so the the first notice uh of injury or loss um or depend depending on on the subdomain. Yes. Um, so depending on the jurisdiction, there's also turnaround time requirements when handling these. So, for example, in Texas, you only have a few hours of processing a claim to get an answer. In other states, it might be 7 days, 21 days that you need to get a response.

43:32 Now, it gets more complicated because if you're a worker who's been working in a factory, let's say for 30 years, you're 60 years old, you're going to have large medical record file, right? So, there's a lot of information here and in the initial report, but to do a full claim analysis, you have to go back and you have to try to pull those. Maybe your name changed because you got married, maybe you moved uh cities, providers, right?

43:55 So you have to reconstruct sort of that that full uh history as well. Uh and I think unfortunately in a lot of litigated cases there's information that's left out uh as well on purpose to due to the fact that a lot of organizations have not digitally transformed and can't process cases fast enough. So uh if this was all that's needed, this would be uh much faster. But unfortunately or more realistically there's a much broader backstory as well.