Question 1

Can a well-designed prompt outperform OpenAI's o1 model?

Accepted Answer

_Human-written prompts can still beat o1's automated reasoning processes._

Yes. A well-designed prompt, meaning the human writes the instruction, can, so far, outpace OpenAI's o1. This is noteworthy because o1 is designed specifically for reasoning through an iterative, self-calling process with dedicated steps for iteration, reflection, and feedback loops. The fact that careful human-written prompts can still exceed its performance suggests that the model's reasoning capabilities, while advanced, have not yet surpassed what skilled human prompt engineering can achieve.

— [Sam Altman Says, 'We Know How To Build AGI' — Not So Fast](https://www.forbes.com/sites/lutzfinger/2025/01/07/sam-altman---agi--ai-workforce-in-2025-the-battle-of-tech-giants/) · _Forbes_

Question 2

Can we conclude that trust causes higher vaccination rates?

Accepted Answer

_Correlation found, but causation works both ways with trust._

While we have shown the impact of trust on covid vaccination, correlations do not indicate causation. Trust is a two-way street. If governments do an excellent job vaccinating their population, it might alter people's overall trust in their institutions. Appropriate, transparent responses will improve the ability of countries to deal with the effects of the COVID pandemic in a virtuous cycle, suggesting the relationship flows both ways.

— [Get Vaccinated? How Trust In Institutions Determines COVID Vaccination Rates In The EU](https://www.forbes.com/sites/lutzfinger/2021/05/06/get-vaccinated-how-trust-into-the-institutions-determines-vaccination-rates-in-the-eu/) · _Forbes_

Question 3

Did you find other factors that influenced vaccination rates besides trust?

Accepted Answer

_Other factors tested but trust stood out statistically._

There are other factors besides trust that might influence vaccination rates, including accessibility and vulnerability. We analyzed the old-age dependency ratio and health care expenditure per inhabitant but did not find statistically significant dependencies. This suggests that while these factors could theoretically matter, in our analysis trust emerged as the key differentiator when controlling for these other variables across EU member states.

— [Get Vaccinated? How Trust In Institutions Determines COVID Vaccination Rates In The EU](https://www.forbes.com/sites/lutzfinger/2021/05/06/get-vaccinated-how-trust-into-the-institutions-determines-vaccination-rates-in-the-eu/) · _Forbes_

Question 4

How did Varo use AI to reduce operational costs while improving customer service?

Accepted Answer

_An AI chatbot cut costs 60% while handling most inquiries._

We started by providing voice support for everybody, but it got very expensive. We moved to live person chat, then built an AI chatbot. That chatbot now contains over 55% of our inbound inquiries and allowed us to reduce operational costs by about 60%. This is part of solving the high-cost problem that made serving underserved populations uneconomical for traditional banks. The AI handles routine questions efficiently while maintaining service quality for customers who value access and responsiveness.

— [Disrupting Traditional Banking, with Colin Walsh](https://www.lutzfinger.com/transcript/2025-05-22-disrupting-traditional-banking-with-colin-walsh/) · _The Keynote on AI_

Question 5

How did you isolate trust as a factor in COVID vaccination rates?

Accepted Answer

_Using the EU to study trust's role in vaccination._

We looked towards the EU because the distribution of vaccines to Member States happened simultaneously and under the same conditions. But not all member states show the same trust, allowing us to isolate trust as a variable. We used the European Social Survey, a bi-annually collected survey of thousands of EU citizens from 29 countries, to measure trust levels. We then correlated that data with vaccination data from the European Centre for Disease Prevention and Control.

— [Get Vaccinated? How Trust In Institutions Determines COVID Vaccination Rates In The EU](https://www.forbes.com/sites/lutzfinger/2021/05/06/get-vaccinated-how-trust-into-the-institutions-determines-vaccination-rates-in-the-eu/) · _Forbes_

Question 6

How did you solve the fairness problem in peer reviews for your Cornell data products course?

Accepted Answer

_Statistical personas identify unfair reviewers in peer grading systems._

We developed a data validation process to ensure fairness in peer reviews. The process analyzes metrics like total review time, time spent on each review, and number of comments left on each project. Most variables followed a Gaussian distribution, letting us use statistical tools to identify problematic reviewers. We built student personas including Lazy Graders, Harsh and Happy Graders, and Misaligned Students. Examining just three to four projects from a single student typically gave us enough insight to determine if they fit these personas, and we would re-evaluate their feedback accordingly.

— [Hack For Fairness: Unlocking The Predictive Power Of Data Validation And Quality Assurance In Education](https://www.forbes.com/sites/lutzfinger/2023/04/03/hack-for-fairness-unlocking-the-predictive-power-of-data-validation-and-quality-assurance-in-education/) · _Forbes_

Question 7

How did you use quizzes to improve the flipped classroom model?

Accepted Answer

_Quizzes revealed where students struggled before class even started._

I asked students to do a quiz before coming to class to find out which areas they didn't understand. For example, a FiveThirtyEight video on k-means clustering Margaritas was fun, but the quiz revealed it failed to explain where the 'k' in k-means came from. This feedback let me spend more time on that point in class than I would have usually done. Content curation became the critical path, and quizzes helped me identify gaps.

— [How COVID Changed Our Classrooms? A Self-Experiment Of Online Teaching.](https://www.forbes.com/sites/lutzfinger/2020/05/14/how-covid-changed-our-classrooms-a-self-experiment-of-online-teaching/) · _Forbes_

Question 8

How do you measure quality in healthcare data?

Accepted Answer

_Quality metrics for healthcare ML._

> Knowing quality is therefore of utmost importance. Three broad measurement areas in today's healthcare: (1) Metrics about quality of care — related to treatment effectiveness and patient satisfaction. (2) Metrics about need and utilization of care — gauging health requirements of a population and how often one uses what type of service. (3) Metrics about value-based healthcare — they measure disease prevention rather than disease…

— [Healthcare And Data - How To Measure Quality](https://www.forbes.com/sites/lutzfinger/2023/06/27/healthcare-and-datahow-to-measure-quality/) · _Forbes_

Question 9

How does Think Labs AI use physics informed AI differently than general purpose models like ChatGPT?

Accepted Answer

_Grid AI must be grounded in physics, not just patterns in data._

We cannot just put SCADA measurements and smart metering information into ChatGPT or Microsoft Copilot or Gemini and ask how the grid works. It does not work like that. We lean into something called Physics Informed AI. We train our AI models using physics. It is physics driven, physics based, and physics accurate. We teach AI how the grid works and reproduce engineering analysis faithfully. The AI can turn traditional studies that take weeks to months into minutes, and go from analyzing one feeder at a time into full system analysis.

— [AI Needs Power, the Grid Needs AI, with Josh Wong](https://www.lutzfinger.com/transcript/2026-04-02-ai-needs-power-the-grid-needs-ai-with-josh-wong/) · _The Keynote on AI_

Question 10

How does your company Gooder AI address the gap between technical metrics and business value?

Accepted Answer

_*Translating predictive model performance into profit and savings estimates*_

Gooder addresses one particular place where there's a need for a new category of software: the metrics. It's a business user console for understanding potential value and making decisions about deployment. Typically, data scientists present technical metrics like area under the curve, precision, or recall, but very rarely go to stakeholders and say if you deploy this model, it has potential value of this much profit or savings. We provide visualization where both dimensions are business relevant, like the decision threshold and business metrics like savings or profit, so stakeholders can see exactly what their options are.

— [The Value Translation Gap: AI's Deployment Problem](https://www.lutzfinger.com/transcript/2024-11-28-the-value-translation-gap-ais-deployment-problem/) · _The Edge_

Question 11

How has the shift from SEO to GEO changed what websites need to do for brands?

Accepted Answer

_Websites now feed AI rather than convert human visitors directly._

Websites used to be designed to convert human visitors, but now their main purpose needs to be feeding data into AI. The three-party model of searcher, Google, and end content is disintegrating into a two-party model. Decisions are getting made before consumers hit the website, so the website is no longer the main conversion point. Instead, websites should be clean data delivery mechanisms to computers, very explicit about who you sell to, your target use case, pain points, and the entire buyer journey. If it's not explicit, the AI has no way to know and might recommend somebody else.

— [Marketing to Humans Changed, with Alex Dees](https://www.lutzfinger.com/transcript/2026-03-24-marketing-to-humans-changed-with-alex-dees/) · _The Keynote on AI_

Question 12

How much higher are conversion rates when using AI-powered search compared to traditional keyword search?

Accepted Answer

_AI search delivers up to 500% higher conversion than keyword search._

I did research comparing traditional keyword search with a more explanatory AI approach. When someone searches for a car like a Honda Odyssey, instead of just showing a list, the AI provides context about the car, options, and nearby dealerships with pricing. The conversion rate from search to buy can be five times higher in some cases. That's a 500% increase. This happens because the chatbot gives personalized recommendations based on knowing the user's full background and context, making the information land much more effectively than traditional search results.

— [Lutz Finger | From SEO to GEO: How AI is Changing How Car Buyers Find You](https://www.lutzfinger.com/transcript/2026-05-22-lutz-finger-from-seo-to-geo-how-ai-is-changing-how-car-buyers-find-you/) · _Three Key Insights_

Question 13

How should product builders think about data bias in AI systems?

Accepted Answer

_Bias in data is inevitable; product decisions determine how much to accept._

There is never a way to have non-biased data, because data is biased. As a product person, you need to decide by how much you want to de-bias the data. Sometimes bias is actually good because you don't want averages, you want the best outcome. For example, if identifying certain patients or optimal machinery output, you don't want what the machine did in the past all the time, but just the best parts of it. You're injecting very good data versus just average data.

— [Building AI products: 5 lessons from our founders' workshop](https://www.lutzfinger.com/transcript/2024-08-01-building-ai-products-5-lessons-from-our-founders-workshop/) · _The Edge_

Question 14

What are the three broad measurement areas in today's healthcare?

Accepted Answer

_Quality, utilization, and value-based prevention are the three areas._

There are three broad measurement areas. First, metrics about quality of care, which relate to treatment effectiveness and patient satisfaction. Second, metrics about need and utilization of care, gauging health requirements of a population and how often one uses what type of service. Third, metrics about value-based healthcare, which measure disease prevention rather than disease treatment, measuring actions taken to prevent or reverse illnesses instead of just treating them.

— [Healthcare And Data - How To Measure Quality](https://www.forbes.com/sites/lutzfinger/2023/06/27/healthcare-and-datahow-to-measure-quality/) · _Forbes_

Question 15

What did Ben learn at LinkedIn about predicting company success with data?

Accepted Answer

_Only founder data predicts early stage success, not company metrics._

Working with the data science team at LinkedIn, we evaluated whether there are signals in the data to predict future company success. After series B, there are signals. Market signals and signals internal to LinkedIn data are predictive. Things like the quality of where you're hiring from, how many people you win compared to how many you lose to other companies. Those are very predictive. In the early stages, no company level data works. The only thing that works is data around founders specifically. This was truly revelating, showing that at the early stages, it's all about the founders.

— [The Hidden Patterns, with Ben Orthlieb](https://www.lutzfinger.com/transcript/2026-04-23-the-hidden-patterns-with-ben-orthlieb/) · _The Keynote on AI_

Question 16

What did the data show about high trust versus low trust countries?

Accepted Answer

_High trust countries vaccinate more, and the gap widens._

We plotted a historical trend of vaccine uptake delineated into high and low trust countries. The plot shows clearly that high trust countries have a higher mean vaccine uptake over low trust countries, with the gap widening over time. We regressed vaccine uptake as of April 18, 2021 with trust in EU institutions. A positive coefficient for trust in EU institutions along with a p-value of less than 0.05 clearly indicates a statistically positive relationship.

— [Get Vaccinated? How Trust In Institutions Determines COVID Vaccination Rates In The EU](https://www.forbes.com/sites/lutzfinger/2021/05/06/get-vaccinated-how-trust-into-the-institutions-determines-vaccination-rates-in-the-eu/) · _Forbes_

Question 17

What does Marpai Connect measure in value-based care, and what do you de-emphasize?

Accepted Answer

_Clinical markers and outcomes matter more than engagement metrics._

At Marpai Connect we use bio markers that are clinically proven, like HBA1C measurements for diabetes. We also use other measured outcomes such as pain reduction scores and functional health improvement scores. We include cost, which is easy for a third-party administrator to measure via claims, but cost alone is not sufficient. What we de-emphasize: engagement metrics, because they often lack evidence that they work. Our approach is working. 40% of the commercial market wants to get access to value-based care.

— [Healthcare And Data - How To Measure Quality](https://www.forbes.com/sites/lutzfinger/2023/06/27/healthcare-and-datahow-to-measure-quality/) · _Forbes_

Question 18

What engagement methods did you test and which worked best?

Accepted Answer

_Student surveys revealed multiple-choice questionnaires won for engagement._

I experimented with cold-calling via a random number generator, multiple-choice questionnaires, discussions in smaller groups, and other methods. After surveying students after each class, multiple-choice questionnaires came out on top. I created them for every 10 minutes of my course to ensure my speed was right and measure my teaching performance. I also used a Google Spreadsheet where students outlined answers in one to two sentences, then picked the most interesting ones to discuss.

— [How COVID Changed Our Classrooms? A Self-Experiment Of Online Teaching.](https://www.forbes.com/sites/lutzfinger/2020/05/14/how-covid-changed-our-classrooms-a-self-experiment-of-online-teaching/) · _Forbes_

Question 19

What has been the impact of implementing data validation in your peer review process?

Accepted Answer

_Instructors now focus time on critical reviews, elevating course quality._

Employing data validation and data quality assurance has been a game-changer for the class. As instructors, we can now concentrate on the most critical reviews, ultimately elevating the course's quality. Instead of reviewing every piece of feedback, we focus our time on re-evaluating feedback from students who fit problematic personas. Our toolset is coded in Python and can be adapted for various courses using FeedbackFruits, making this approach scalable beyond just our Designing Data Products course.

— [Hack For Fairness: Unlocking The Predictive Power Of Data Validation And Quality Assurance In Education](https://www.forbes.com/sites/lutzfinger/2023/04/03/hack-for-fairness-unlocking-the-predictive-power-of-data-validation-and-quality-assurance-in-education/) · _Forbes_

Question 20

What is LIME and how does it help with explainable AI?

Accepted Answer

_LIME removes pixels to reveal what triggers AI decisions._

LIME is a tool designed to help with explainability in AI systems, particularly for complex models like deep learning that create their own variables which might not make immediate sense for humans. LIME works by creating many different versions of the same image through removing pixels. This process helps researchers and users understand what specifically triggered a given AI response. It addresses the challenge that while decision trees, Bayesian classifiers, and linear models are easy to interpret, deep learning models cannot be explained so easily.

— [President Biden Is Man, Woman And 40 Years Old - Why We Need Algorithmic Transparency](https://www.forbes.com/sites/lutzfinger/2021/02/04/president-biden-is-man-woman-and-40-years-old---why-we-need-algorithmic-transparency/) · _Forbes_

Question 21

What is the 'Ask, Measure, Learn' framework?

Accepted Answer

_The framework from Lutz's book._

> "Ask Measure Learn" is a non-technical guide shows you how to extract significant business value from big data. It is a system that helps you ask the right questions, measure the right data, and then learn from the results. The framework was originally devised to help governments and NGOs sift through volumes of data.

— [Ask Measure Learn](https://www.lutzfinger.com/book/) · _Book — Ask, Measure, Learn_

Question 22

What is the AI Effect and how does it relate to AGI?

Accepted Answer

_AGI's definition keeps shifting as AI capabilities advance over time._

The AI Effect describes how the definition of AGI often shifts as AI systems become capable of performing tasks once thought to require general intelligence. Some colleagues joke that AGI is 'what we haven't yet built.' Altman recently downplayed the G in AGI, saying his guess is we will hit AGI sooner than most people think, and it will matter much less. Just on Sunday, he said it has become a very sloppy term. This sounds like moving the goalpost.

— [Sam Altman Says, 'We Know How To Build AGI' — Not So Fast](https://www.forbes.com/sites/lutzfinger/2025/01/07/sam-altman---agi--ai-workforce-in-2025-the-battle-of-tech-giants/) · _Forbes_

Question 23

What is the book Ask, Measure, Learn about?

Accepted Answer

_Topics covered in the O'Reilly book._

> **Publisher:** O'Reilly Media **Published:** Feb 1, 2014

— [Ask Measure Learn](https://www.lutzfinger.com/book/) · _Book — Ask, Measure, Learn_

Question 24

What is the difference between virality and contagiousness?

Accepted Answer

_Stories gain weight as they travel, not uniform infection rates._

Virality is something which is equally infectious at each stage. If I have a story which one out of 10 people like, that story doesn't change. One out of 10 people like it in the future as well. Contagiousness is different. As long as it travels, it gains weight. The more people believe in it, the more trustworthy the story becomes. Even a wrong information becomes more believed the longer it travels. At some point everyone believes it's truth no matter what because so many people believed in it. Forget about virality. The right word is contagiousness.

— [Measuring Social Media What Marketers Need To Know](https://www.lutzfinger.com/transcript/2014-08-22-measuring-social-media-what-marketers-need-to-know/) · _Social Media Examiner_

Question 25

What is value-based care and how does AI enable it?

Accepted Answer

_VBC framing and AI's role._

> Healthcare is on the brink of a seismic shift, and AI is its catalyst. Three converging trends are creating a perfect opportunity for change: (1) Foundational Models' growing ability to predict risks further into the future compared to traditional machine learning, (2) the rise of the self-insured healthcare industry motivated to address these risks, and (3) a surge in Value-Based Care (VBC) providers focused on risk reduction. These…

— [Three Trends Of Transformation: How AI & Value-Based Care (VBC) Will Drive Transformation Of Self-Insured Healthcare](https://www.forbes.com/sites/lutzfinger/2023/05/01/three-trends-of-transformation-how-ai--value-based-care-vbc-will-drive-transformation-of-self-insured-healthcare/) · _Forbes_

Question 26

What makes achieving general intelligence fundamentally different from doing many specialized tasks well?

Accepted Answer

_Generalization beyond training data defines AGI, not task breadth alone_

The key difference is the learning and generalization capability, not breadth of tasks. A system that does 95% of human jobs by copying millions of training examples is not AGI, even if economically valuable. True AGI must take a leap beyond what it was prepared for and generalize from what it knows to do new things. If you describe a novel alien civilization with bizarre customs never in training data, and the system reasons about it from multiple perspectives, that shows real inductive and abductive reasoning. AGI should figure out how to achieve unfamiliar goals in unfamiliar environments through creative imagination and trial and error, not just pattern matching against a massive catalog of prior examples.

— [Rethinking the AGI Race, with Benjamin Goertzel](https://www.lutzfinger.com/transcript/2026-05-28-rethinking-the-agi-race-with-benjamin-goertzel/) · _The Keynote on AI_

Question 27

What metrics do you use for data validation in peer review quality assurance?

Accepted Answer

_Review time and comment patterns reveal data quality issues._

Our data validation process is based on total review time, time spent on each review, the number of comments left on each project, and more. Most variables followed a Gaussian distribution, allowing us to use various statistical tools similar to those that identified cheating sumo wrestlers or overlooked baseball players. Based on predicted values from these metrics, we can identify problematic reviewing patterns. This approach mirrors data quality assurance in machine learning, where high-quality labeled data leads to better model performance.

— [Hack For Fairness: Unlocking The Predictive Power Of Data Validation And Quality Assurance In Education](https://www.forbes.com/sites/lutzfinger/2023/04/03/hack-for-fairness-unlocking-the-predictive-power-of-data-validation-and-quality-assurance-in-education/) · _Forbes_

Question 28

What outcome metrics does Marpai Connect use to evaluate care effectiveness?

Accepted Answer

_Specific metrics track adherence, improvement, and patient participation._

My team at Marpai designed outcome metrics with our value-based care partners to evaluate the effectiveness of care. These include medication adherence, functional health improvement, flu vaccination rate, and patient engagement. We also built financial alignment into the platform, where providers should only receive payment if they effectively help reduce costs and improve health on average. For example, NuvoAir committed to a total cost of care reduction of 35% based on claims analysis with their virtual-first care model.

— [Changing Healthcare - A Platform Approach Using Data & AI](https://www.forbes.com/sites/lutzfinger/2023/07/05/changing-healthcarea-platform-approach-using-data--ai/) · _Forbes_

Question 29

What percentage of healthcare costs are linked to reversible or manageable diseases?

Accepted Answer

_Nearly half of healthcare spending could be avoided with better care._

At Marpai Inc., we discovered that 23% of all healthcare costs are linked to reversible diseases. An additional 18% could remain stable with better management. That's a staggering 42% of essentially avoidable costs. This finding highlights the enormous opportunity to reduce healthcare spending by focusing on prevention and better disease management rather than just treating illness.

— [Changing Healthcare - A Platform Approach Using Data & AI](https://www.forbes.com/sites/lutzfinger/2023/07/05/changing-healthcarea-platform-approach-using-data--ai/) · _Forbes_

Question 30

What unintended consequences can quality metrics create in healthcare?

Accepted Answer

_Mortality metrics can discourage doctors from treating complex cases._

As Michael Howell reminded me, the real challenges are in finding the metrics that matter and preventing unintended consequences with any given metric. Take a simple metric on mortality for surgery: patients should not die during surgery. But what if the patient's situation was complex to start with? There are cases where a patient did not receive care and died because giving him appropriate care could have resulted in a worse quality score for the doctor.

— [Healthcare And Data - How To Measure Quality](https://www.forbes.com/sites/lutzfinger/2023/06/27/healthcare-and-datahow-to-measure-quality/) · _Forbes_

Question 31

What would Clubhouse need to do to understand conversations and improve content discovery?

Accepted Answer

_Live transcription is technically possible but not currently implemented._

To find the right discussion for you, Clubhouse would need to understand all conversations, meaning it would need to live-transcribe them. Technically this is feasible. Speech to text algorithms have become very powerful. Clubhouse is currently not transcribing conversations, presumably to preserve the authenticity of the moment. However, streaming companies like Spotify or Deezer can tell you that a delay between the user's dislike and an action such as skip makes it harder for the algorithm to work correctly. Clubhouse will need data and engagement metrics to scale.

— [Clubhouse's Future Depends On Data - How To Build A TikTok Like Algorithm](https://www.forbes.com/sites/lutzfinger/2021/02/09/clubhouses-future-depends-on-datahow-to-build-a-tiktok-like-algorithm/) · _Forbes_

Question 32

Why do AI models still need human involvement even with unsupervised learning?

Accepted Answer

_Even advanced models can't escape the need for human judgment._

We need humans because there isn't always a true north for what's correct. Supervised learning works when there's a clear label, like Obama was president. That's a fact. But when you get into perception, like whether a joke is funny or which word to use about Obama's policies, it depends on context and perspective. The same happens with generated images. Art often creates things society didn't expect, so at what point is something random versus amazing? Since there's not a true label for certain things, we constantly need people to give feedback on the model and provide guardrails.

— [AI and the human factor](https://www.lutzfinger.com/transcript/2023-07-07-ai-and-the-human-factor/) · _The Edge_

Question 33

Why do LLMs convert better than search?

Accepted Answer

_Trust, personalization, and conversion lift from AI traffic._

> Some predict that by 2028, more people will discover products and information through large language models (LLMs) like ChatGPT and Gemini than through traditional search engines. But based on research I conducted with Cornell Master's students, that shift is happening much faster. LLM-driven traffic is already starting to outperform traditional search — not in volume, but in value.

— [Study Shows LLM Conversion Rate Is 9x Better — AEO Is Coming](https://www.forbes.com/sites/lutzfinger/2025/06/19/study-shows-llm-conversion-rate-is-9x-better---aeo-is-coming/) · _Forbes_

Question 34

Why does Clubhouse's future success depend on using user data?

Accepted Answer

_Finding content on Clubhouse is hard without better data use._

Clubhouse's future success depends on how it can help users find content and enable engagement through the use of user data. At the core, any social app solves two issues: to find and enable social interactions. Currently, finding interesting conversations on Clubhouse is not easy. You curate your own content by following people, similar to how Twitter used the social graph as a first approximation for your interest. A social graph is a clunky way to curate your feed. In order to scale, Clubhouse will need data and engagement metrics.

— [Clubhouse's Future Depends On Data - How To Build A TikTok Like Algorithm](https://www.forbes.com/sites/lutzfinger/2021/02/09/clubhouses-future-depends-on-datahow-to-build-a-tiktok-like-algorithm/) · _Forbes_

Question 35

Why is eval data a moat?

Accepted Answer

_Evaluation data as the strategic differentiator._

> All of them matter, and all of them are being fought over. But the real value may sit in the one thing OpenAI, Anthropic, and Gemini have not yet made central: your own eval data. Sure, there is the occasional thumbs-up button. But who is really using that signal?

— [The Missing Moat In AI: Your Eval Data](https://www.forbes.com/sites/lutzfinger/2026/05/26/the-missing-moat-in-ai-your-eval-data/) · _Forbes_

Question 36

Why is it important to measure appropriateness of medical interventions, not just quality?

Accepted Answer

_Fee-for-service payment can incentivize unnecessary procedures._

As Marty Makary told me, it is not sufficient to just measure the quality of medical procedures, but we also need to evaluate the appropriateness of these interventions. The current fee-for-service payment structure might encourage some doctors to perform multiple surgeries rather than one, as they are paid per procedure. Transparency in correcting misalignments is key. This is why utilization metrics look at whether someone should have gotten a given intervention in the first place.

— [Healthcare And Data - How To Measure Quality](https://www.forbes.com/sites/lutzfinger/2023/06/27/healthcare-and-datahow-to-measure-quality/) · _Forbes_

Question 37

Why is it so hard to know the quality of healthcare we receive?

Accepted Answer

_Patients lack data because they are not the customer._

We rate Uber drivers, restaurants, and Amazon delivery, but when it comes to our own health experience, we have little to no data. The reason is structural: the business model of the healthcare provider is to get money from the insurance company, not the patient. The patient is not the customer. Because of this misalignment, knowing quality is of utmost importance, yet the information is not readily available to us as patients.

— [Healthcare And Data - How To Measure Quality](https://www.forbes.com/sites/lutzfinger/2023/06/27/healthcare-and-datahow-to-measure-quality/) · _Forbes_

Question 38

Why is measuring social media so difficult despite all the tools available today?

Accepted Answer

_Tools are everywhere, but strategy comes first, not scores._

The problem is that measurement used to require specialists with big databases and tooling. Now with social media, you suddenly have all these tools that give you cloud scores and metrics instantly. People see a score and decide something based on it without planning what they actually want to achieve. If you haven't formulated your business goal first, any measurement won't work. People start with the measurement and then think about what to do with it, while the correct way is the opposite: start with a question, then come up with a measure.

— [Measuring Social Media What Marketers Need To Know](https://www.lutzfinger.com/transcript/2014-08-22-measuring-social-media-what-marketers-need-to-know/) · _Social Media Examiner_

Question 39

Why is workflow integration important for OpenAI's feedback loops?

Accepted Answer

_Integration into workflows helps measure answer quality and drives specialization._

Feedback loops work best if they are integral to an existing workflow. We will see more and more workflows become supercharged with LLMs. Once OpenAI is integrated into those workflows, it's easier to see when and whether a user accepts a change or suggestion. OpenAI will need to invest more efforts into workflows and products to be more than just a generic language interface. Specialization will be the norm. Initially, OpenAI used the fast route by tagging along with Microsoft, with Copilot offering integrations into many of Microsoft's core tools.

— [How OpenAI Can Secure Its Investments - A Playbook That Needs Microsoft](https://www.forbes.com/sites/lutzfinger/2023/08/18/the-potential-moat-of-openaihow-openai-can-secure-its-investments/) · _Forbes_

Q&A, Measurement

Can a well-designed prompt outperform OpenAI's o1 model?

Can we conclude that trust causes higher vaccination rates?

Did you find other factors that influenced vaccination rates besides trust?

How did Varo use AI to reduce operational costs while improving customer service?

How did you isolate trust as a factor in COVID vaccination rates?

How did you solve the fairness problem in peer reviews for your Cornell data products course?

How did you use quizzes to improve the flipped classroom model?

How do you measure quality in healthcare data?

How does Think Labs AI use physics informed AI differently than general purpose models like ChatGPT?

How does your company Gooder AI address the gap between technical metrics and business value?

How has the shift from SEO to GEO changed what websites need to do for brands?

How much higher are conversion rates when using AI-powered search compared to traditional keyword search?

How should product builders think about data bias in AI systems?

What are the three broad measurement areas in today's healthcare?

What did Ben learn at LinkedIn about predicting company success with data?

What did the data show about high trust versus low trust countries?

What does Marpai Connect measure in value-based care, and what do you de-emphasize?

What engagement methods did you test and which worked best?

What has been the impact of implementing data validation in your peer review process?

What is LIME and how does it help with explainable AI?

What is the 'Ask, Measure, Learn' framework?

What is the AI Effect and how does it relate to AGI?

What is the book Ask, Measure, Learn about?

What is the difference between virality and contagiousness?

What is value-based care and how does AI enable it?

What makes achieving general intelligence fundamentally different from doing many specialized tasks well?

What metrics do you use for data validation in peer review quality assurance?

What outcome metrics does Marpai Connect use to evaluate care effectiveness?

What percentage of healthcare costs are linked to reversible or manageable diseases?

What unintended consequences can quality metrics create in healthcare?

What would Clubhouse need to do to understand conversations and improve content discovery?

Why do AI models still need human involvement even with unsupervised learning?

Why do LLMs convert better than search?

Why does Clubhouse's future success depend on using user data?

Why is eval data a moat?

Why is it important to measure appropriateness of medical interventions, not just quality?

Why is it so hard to know the quality of healthcare we receive?

Why is measuring social media so difficult despite all the tools available today?

Why is workflow integration important for OpenAI's feedback loops?