2023-03-30 AI Committee Meeting Minutes

Public Page

Date

Mar 30, 2023

Agenda

• Welcome/Networking

• Antitrust

  • This meeting is subject to the terms of the anti-trust and it will be recorded.

• Meeting minute Review

• Accuracy Calculations & Expectation Managment

Meeting Minutes

• Antitrust Accepted

• Meeting Minutes Reviewed Accepted

• Meeting minutes

  • Last meeting, we were technical and looked at the payloads, but this meeting we want to look at the prerequisites of AI.

  • Raj went over a document that is attached below.

    • Document explains what kind of data input should be sent for AI so it can be compatible with most of the AI companies. What kind of impact does the data prerequisites have on the accuracies of the AI. Hoping this White Page can be documented for the CIECA website. This can help set the standardization into the data inputs and the payloads, which also get into the integration.

    • What are the expectancies of the AI accuracies?

      • Different Use Cases and their accuracy calculations are different and one of the reasons that AI is being adopted slowly in this industry as in comparison with other industries.

      • AI can be 98% accurate, but what does that 98% mean or include?

      • The carriers are looking at the accuracies, their accuracy calculations and KPS are totally different. So base this on a one of the one of the education expectations management and also set the ground rule when it comes to how everybody should be looking at the accuracy trough the same mindset is going to be easier for the AI companies.

        • We could use a form to discuss the mindset of the AI accuracies between carriers and the collision repair shops and everyone else using AI in the industry. We need to discuss the use cases and the inputs that everyone gets so we can see the KPIs and divide them into multiple categories.

          • AI identifying parts would be a category. Is this 10% accurate or 90% accurate? How good are identifying all the different parts that are visible in the picture?

          • A second category of Ran Damages

          • category identifying the length of the damage, the depth of the damage, again working the accuracy metrics.

          • Severity analysis and the repair and replace analysis.

          • Another is unrelated prior damages.

        • Is the estimate in the case accurate or inaccurate, or how much percentage is inaccurate? These are the different ways the customer sees the results and what is causing the slow adoption of AI.

      • Terminology needs to be standardized.

        • We need definitions so the evaluation metrics that defines the KPI’s of how things are done.

        • When looking at an AI estimate, there may be 10 lines on the estimate, if one is wrong, it is marked as inaccurate, which means the estimate was 90% correct, but one line caused it to be seen as wrong.

        • How quick can an estimate be generated? The time of an AI estimate is different based on severity. However, the AI estimate saves 50% to 50% of the time the adjuster manually creates an estimate.

        • The AI is very accurate with minor damage; however, it is less accurate the more damage is done to the vehicle.

    • We want to make sure to respect that all AI companies have proprietary tools and algorithms.

      • We can talk about the expectations of AI at a high level, but each company will continue to train and build the AI algorithms to make their product more accurate.

      • At this time even though we are competitors in the industry, we need to come together to help educate the industry.

    • Use Cases need to be built on how AI is going to be used in the industry. If we can star documenting for each business case and the factors to listen to each of them to populate how we build a standard framework which will help everyone.

      • Triage/FNOL

      • Estimate

      • Auditing

      • Reviewing

    • Carriers and the customers will have different solutions that will be able to use a same scale to measure damage. This is the education that we need to bring.

      • Raj will volunteer to take a look and generate the use cases and then we can work on them as a committee.

      • Jimmy has also been working on use cases that we can put together to show to different solutions.

    • Expectation management and education around the accuracies we are talking about, right, not every single thing can be put it into the definition or a standard kind of an it's not a part name, right. It's a very generic kind of in topic. So, I don't know how CIECA wants to basically standardize it or basically document it so that everybody can get educated and then set their expectations about AI.

Up Next

• Welcome/Networking

• Antitrust

• Meeting minute Review

Action items

Decisions

Participants

• @Paulette Reed

• Ginny Whelan

• Pete Sheehan

• Stacy Phillips

• Raj Pofale

• Nikki Lerol

• Kim Sorlien

• Abhijeet Gulati

• Paul Barry

• Mark Ficher

• Phil Martinez

• William Brower

• Julie Massaro

• Gene Lopez

• Don Porter

• Charles Sulkala

• James Spears

Participants in the meetings are noted for your information.  If you have questions on the committee’s activities, please contact a recent attendee. https://cieca.atlassian.net/wiki/spaces/CPSC/pages/1371340801