Question 1

Did you find other factors that influenced vaccination rates besides trust?

Accepted Answer

_Other factors tested but trust stood out statistically._

There are other factors besides trust that might influence vaccination rates, including accessibility and vulnerability. We analyzed the old-age dependency ratio and health care expenditure per inhabitant but did not find statistically significant dependencies. This suggests that while these factors could theoretically matter, in our analysis trust emerged as the key differentiator when controlling for these other variables across EU member states.

— [Get Vaccinated? How Trust In Institutions Determines COVID Vaccination Rates In The EU](https://www.forbes.com/sites/lutzfinger/2021/05/06/get-vaccinated-how-trust-into-the-institutions-determines-vaccination-rates-in-the-eu/) · _Forbes_

Question 2

How did LinkedIn use knowledge graphs to solve the job title problem?

Accepted Answer

_Structuring 150 million job titles required more than simple categorization._

LinkedIn had over 150 million unique job title strings, which was impossible to model effectively. We decomposed titles into four facets: seniority, employment status, role, and specialty. For example, Senior Software Engineer contains seniority information, a role, and a specialty. We then built knowledge graphs for each dimension to show how values interrelated. This helped us understand completeness and vagueness of titles. A VP at a bank means something completely different than a VP at a tech company. Taxonomies alone were not expressive enough to capture these nuances.

— [The synergy between LLMs and knowledge graphs](https://www.lutzfinger.com/transcript/2023-08-24-the-synergy-between-llms-and-knowledge-graphs/) · _The Edge_

Question 3

How do you measure quality in healthcare data?

Accepted Answer

_Quality metrics for healthcare ML._

> Knowing quality is therefore of utmost importance. Three broad measurement areas in today's healthcare: (1) Metrics about quality of care — related to treatment effectiveness and patient satisfaction. (2) Metrics about need and utilization of care — gauging health requirements of a population and how often one uses what type of service. (3) Metrics about value-based healthcare — they measure disease prevention rather than disease…

— [Healthcare And Data - How To Measure Quality](https://www.forbes.com/sites/lutzfinger/2023/06/27/healthcare-and-datahow-to-measure-quality/) · _Forbes_

Question 4

How do you spot social media bots?

Accepted Answer

_Detection heuristics._

> Figure 6-4. Subset of a social network, created using LinkedIn Labs Since bot networks will first create their own group by linking to one another before they start to infiltrate the rest of the network, especially in the beginning, you will see a highly dense network with very low cohesion to the rest of the network. However, those measures using network topology only work in the beginning as the bot network is formed. Once the bots…

— [Finger Dutta Ask Measure Learn (1)](https://www.lutzfinger.com/book/) · _Book — Ask, Measure, Learn_

Question 5

How does Marpai Connect use AI to overcome the small numbers problem?

Accepted Answer

_Pooling clients and predictive AI create scale for small employers._

Marpai Connect addresses the small numbers challenge by pooling all self-insured clients and anticipating an overall reduction in aggregate. We use various approaches from simple time series modeling to advanced deep learning models like Neural Prophet to predict risk based on longitudinal claims records. This pooling strategy allows us to achieve the statistical power that individual small employers lack, while still delivering personalized care through AI matchmaking that pairs each user with the most suitable value-based care partner.

— [Changing Healthcare - A Platform Approach Using Data & AI](https://www.forbes.com/sites/lutzfinger/2023/07/05/changing-healthcarea-platform-approach-using-data--ai/) · _Forbes_

Question 6

How does Richard distinguish between jobs that will be automated by AI and those that won't?

Accepted Answer

_Data availability determines which jobs AI will transform or replace first._

Richard says you can predict how much an industry will change based on how much data is available for a given task. Jobs where outputs are publicly available on the internet, especially all-digital jobs, are where AI can learn from humanity's collective knowledge and automate the most. Plumbing, for example, has no automation potential because no one is collecting data about what plumbers do. Jobs with abundant digital data and public outputs face the highest automation risk in the coming years.

— [The Future of AI, with Richard Socher](https://www.lutzfinger.com/transcript/2026-02-05-the-future-of-ai-with-richard-socher/) · _The Keynote on AI_

Question 7

How does TikTok's algorithm differ from Clubhouse's approach to content discovery?

Accepted Answer

_TikTok uses behavior and content analysis, not social graphs._

TikTok is an algorithm-first company. Unlike Clubhouse or Facebook, it does not mainly rely on the social graph but has built a powerful recommendation engine. TikTok's machine learning magic is a supervised learning system that finds videos interesting to the user by measuring how long you look, how fast you scroll, and many other behavioral features. Additionally, the algorithm evaluates the actual video clip: content, speed, sound, colors, words. Bytedance has an absurd proportion of their software engineers focused on their algorithms. This data has created a sizable moat that not even Facebook could cross easily.

— [Clubhouse's Future Depends On Data - How To Build A TikTok Like Algorithm](https://www.forbes.com/sites/lutzfinger/2021/02/09/clubhouses-future-depends-on-datahow-to-build-a-tiktok-like-algorithm/) · _Forbes_

Question 8

What are the three student personas you identified through data validation in peer reviews?

Accepted Answer

_Three types of problematic reviewers emerged from our validation process._

We identified three distinct student personas. Lazy Graders select largely the same answer for all questions regardless of whether responses are warranted, or spend minimal time on reviews. Harsh and Happy Graders are students who were unexpectedly strict or lenient on one project compared to others who graded that same project. Misaligned Students consistently rate all their projects significantly more positively or negatively compared to the rest of the class. These personas help us identify which feedback needs re-evaluation to maintain fairness.

— [Hack For Fairness: Unlocking The Predictive Power Of Data Validation And Quality Assurance In Education](https://www.forbes.com/sites/lutzfinger/2023/04/03/hack-for-fairness-unlocking-the-predictive-power-of-data-validation-and-quality-assurance-in-education/) · _Forbes_

Question 9

What did Ben learn at LinkedIn about predicting company success with data?

Accepted Answer

_Only founder data predicts early stage success, not company metrics._

Working with the data science team at LinkedIn, we evaluated whether there are signals in the data to predict future company success. After series B, there are signals. Market signals and signals internal to LinkedIn data are predictive. Things like the quality of where you're hiring from, how many people you win compared to how many you lose to other companies. Those are very predictive. In the early stages, no company level data works. The only thing that works is data around founders specifically. This was truly revelating, showing that at the early stages, it's all about the founders.

— [The Hidden Patterns, with Ben Orthlieb](https://www.lutzfinger.com/transcript/2026-04-23-the-hidden-patterns-with-ben-orthlieb/) · _The Keynote on AI_

Question 10

What did Lutz warn about China's AI strategy back in 2019, and why?

Accepted Answer

_Data collection strategy, not just tech, drives AI advantage._

Back in 2019, I warned that China's aggressive data collection strategy would give it a long-term AI advantage over the U.S. The concern was not about computing power or talent, but about data access. China's willingness to collect data aggressively creates a structural advantage because, as Lutz argues elsewhere in the piece, the real question is who controls the data and who builds products people actually use, not just who has the best technology.

— [Stargate: The $500 Billion Message To China — The Future Of AI Regulation](https://www.forbes.com/sites/lutzfinger/2025/01/22/stargate-the-500b-message-to-china---the-future-of-ai-regulation/) · _Forbes_

Question 11

What did the data show about high trust versus low trust countries?

Accepted Answer

_High trust countries vaccinate more, and the gap widens._

We plotted a historical trend of vaccine uptake delineated into high and low trust countries. The plot shows clearly that high trust countries have a higher mean vaccine uptake over low trust countries, with the gap widening over time. We regressed vaccine uptake as of April 18, 2021 with trust in EU institutions. A positive coefficient for trust in EU institutions along with a p-value of less than 0.05 clearly indicates a statistically positive relationship.

— [Get Vaccinated? How Trust In Institutions Determines COVID Vaccination Rates In The EU](https://www.forbes.com/sites/lutzfinger/2021/05/06/get-vaccinated-how-trust-into-the-institutions-determines-vaccination-rates-in-the-eu/) · _Forbes_

Question 12

What is a data product?

Accepted Answer

_Definition and how to build one._

> Now, to be a good product developer, you need to think about self-referential loops within your products. How to get signal, how to get feedback from your users so your AI, or your data science, can leverage that information to make the product work better. So that moved a little bit within it. The innovation, and the inspiration about, and the curiosity about how to use data in novel manners, that's still data science, because that's…

— [Unit 3 Full Transcript](https://www.lutzfinger.com/ecornell/) · _eCornell course_

Question 13

What is a knowledge graph and how does it differ from a taxonomy?

Accepted Answer

_Knowledge graphs connect concepts across dimensions, unlike simple taxonomies._

A knowledge graph is a collection of facts interconnected in a network, storing information as concepts rather than just labels or numbers. It works like a smarter database that imitates human memory. A taxonomy is much simpler, focusing on one characteristic like what kind of thing something is. Elephants are mammals in a taxonomy. But you cannot relate entities of different types. A knowledge graph lets you express that leopards eat gazelles or elephants eat bamboo, connecting different taxonomies together. It is essentially a multi-dimensional, supercharged taxonomy approach.

— [The synergy between LLMs and knowledge graphs](https://www.lutzfinger.com/transcript/2023-08-24-the-synergy-between-llms-and-knowledge-graphs/) · _The Edge_

Question 14

What is the 'Ask, Measure, Learn' framework?

Accepted Answer

_The framework from Lutz's book._

> "Ask Measure Learn" is a non-technical guide shows you how to extract significant business value from big data. It is a system that helps you ask the right questions, measure the right data, and then learn from the results. The framework was originally devised to help governments and NGOs sift through volumes of data.

— [Ask Measure Learn](https://www.lutzfinger.com/book/) · _Book — Ask, Measure, Learn_

Question 15

What is the book Ask, Measure, Learn about?

Accepted Answer

_Topics covered in the O'Reilly book._

> **Publisher:** O'Reilly Media **Published:** Feb 1, 2014

— [Ask Measure Learn](https://www.lutzfinger.com/book/) · _Book — Ask, Measure, Learn_

Question 16

What is the difference between virality and contagiousness?

Accepted Answer

_Stories gain weight as they travel, not uniform infection rates._

Virality is something which is equally infectious at each stage. If I have a story which one out of 10 people like, that story doesn't change. One out of 10 people like it in the future as well. Contagiousness is different. As long as it travels, it gains weight. The more people believe in it, the more trustworthy the story becomes. Even a wrong information becomes more believed the longer it travels. At some point everyone believes it's truth no matter what because so many people believed in it. Forget about virality. The right word is contagiousness.

— [Measuring Social Media What Marketers Need To Know](https://www.lutzfinger.com/transcript/2014-08-22-measuring-social-media-what-marketers-need-to-know/) · _Social Media Examiner_

Question 17

What is the potential financial impact of addressing Type 2 Diabetes in self-insured populations?

Accepted Answer

_Type 2 Diabetes costs are avoidable with right interventions._

Looking at Marpai's data, about 6% of members have Type 2 Diabetes, costing self-insured clients 35 million USD. The key insight is that Type 2 Diabetes can be reversed and these costs are avoidable. This represents a significant opportunity for self-insured companies. By using AI and data to match the best intervention to the best use case, we can maximize therapeutic success and improve ROI. At Marpai we evaluated over 60 different healthcare programs to partner with, specifically to address conditions like this where costs are preventable.

— [Three Trends Of Transformation: How AI & Value-Based Care (VBC) Will Drive Transformation Of Self-Insured Healthcare](https://www.forbes.com/sites/lutzfinger/2023/05/01/three-trends-of-transformation-how-ai--value-based-care-vbc-will-drive-transformation-of-self-insured-healthcare/) · _Forbes_

Question 18

What lesson did you learn from private equity's data acquisition spree in the mid-2010s?

Accepted Answer

_Raw data alone didn't justify bigger multiples back then._

In the mid-2010s, private equity funds went on a purchasing spree with a simple thesis: merge data assets and sell at bigger multiple. I was skeptical. In my book Ask, Measure, Learn, I cautioned that raw data without a focus on impact is just noise. You can plot a line with two points; a million more won't raise valuation. Transformers will flip that script. Now, with generative AI, the situation is fundamentally different for private equity.

— [AI For Private Equity — The Future Of B2B Distribution](https://www.forbes.com/sites/lutzfinger/2025/07/15/ai-for-private-equity---the-future-of-b2b-distribution/) · _Forbes_

Question 19

What metrics do you use for data validation in peer review quality assurance?

Accepted Answer

_Review time and comment patterns reveal data quality issues._

Our data validation process is based on total review time, time spent on each review, the number of comments left on each project, and more. Most variables followed a Gaussian distribution, allowing us to use various statistical tools similar to those that identified cheating sumo wrestlers or overlooked baseball players. Based on predicted values from these metrics, we can identify problematic reviewing patterns. This approach mirrors data quality assurance in machine learning, where high-quality labeled data leads to better model performance.

— [Hack For Fairness: Unlocking The Predictive Power Of Data Validation And Quality Assurance In Education](https://www.forbes.com/sites/lutzfinger/2023/04/03/hack-for-fairness-unlocking-the-predictive-power-of-data-validation-and-quality-assurance-in-education/) · _Forbes_

Question 20

What was Ultimate AI's key insight that made customer onboarding faster than competitors?

Accepted Answer

_Automated clustering replaced manual training data creation for faster setup_

We couldn't afford AI trainers manually creating training data, so we built a clustering pipeline that analyzed historical customer conversations already sitting in CRMs like Salesforce. This let us automatically find the 100 or 200 most common questions and show customers all 30,000 actual ways users asked questions like where's my order. Competitors were using FAQ pages and having people manually write 50 to 100 variations, which didn't reflect real customer language. Our approach gave faster time to value because we used actual conversation data instead of imagined variations, and customers could see their real top questions before even talking to us.

— [How Ultimate evolved from hundreds of supervised models to UltimateGPT](https://www.lutzfinger.com/transcript/2023-05-12-how-ultimate-evolved-from-hundreds-of-supervised-models-to-ultimategpt/) · _The Edge_

Question 21

What would Clubhouse need to do to understand conversations and improve content discovery?

Accepted Answer

_Live transcription is technically possible but not currently implemented._

To find the right discussion for you, Clubhouse would need to understand all conversations, meaning it would need to live-transcribe them. Technically this is feasible. Speech to text algorithms have become very powerful. Clubhouse is currently not transcribing conversations, presumably to preserve the authenticity of the moment. However, streaming companies like Spotify or Deezer can tell you that a delay between the user's dislike and an action such as skip makes it harder for the algorithm to work correctly. Clubhouse will need data and engagement metrics to scale.

— [Clubhouse's Future Depends On Data - How To Build A TikTok Like Algorithm](https://www.forbes.com/sites/lutzfinger/2021/02/09/clubhouses-future-depends-on-datahow-to-build-a-tiktok-like-algorithm/) · _Forbes_

Question 22

Who does Lutz expect will be Clubhouse's fiercest competitors and why?

Accepted Answer

_Twitter and Facebook have social graphs and content spread measurement._

I expect the fiercest competitors to be Twitter and Facebook, as they bring existing social network graphs and also measure how content spreads. As Clubhouse grows, their design choices will be necessary. Whether or not Clubhouse will become a super-aggregator will depend on their ability to create a multi-sided network with decreasing acquisition costs. In order to scale, they will need data and engagement metrics. The trend Clubhouse started is undoubtedly not a covid19 fad. It's the unbundling of content creation and distribution.

— [Clubhouse's Future Depends On Data - How To Build A TikTok Like Algorithm](https://www.forbes.com/sites/lutzfinger/2021/02/09/clubhouses-future-depends-on-datahow-to-build-a-tiktok-like-algorithm/) · _Forbes_

Question 23

Why did you choose e-commerce to study generative AI adoption rather than other industries?

Accepted Answer

_E-commerce leads AI adoption, making it the ideal test case._

In my online certificate programs, I've explored AI use cases across healthcare, media, coffee, finance and e-commerce. Healthcare tends to be slowest to adopt AI, while e-commerce consistently leads. E-commerce already uses AI for targeting ads and emails, product recommendations, and SEO. Since e-commerce leads in AI adoption generally, I assumed they would readily embrace generative AI. That's why I chose to analyze this industry at ShopTalk, expecting to see the generative AI future already happening there if anywhere.

— [ShopTalk 2025: Just 1% Of Vendors Use Generative AI To The Max](https://www.forbes.com/sites/lutzfinger/2025/03/21/state-of-genai--only-1---shoptalk-2025-in-numbers/) · _Forbes_

Question 24

Why did you mention Google's 2008 prediction capabilities and WIRED's 'end of theory' article?

Accepted Answer

_History shows prediction capability doesn't eliminate need for theory._

In 2008, excited from Google's ability to predict the next click, WIRED magazine proclaimed 'the end of theory,' saying that all that was needed is data. That proved to be wrong. Now we have better models for the same approach with transformers doing next-token prediction, but we still have not seen the end of theory. This parallel shows that impressive prediction capabilities, whether for clicks or tokens, don't replace the need for underlying theory and understanding. It's a reminder that we've been at this place before with prediction hype.

— [Healthcare's AI Lesson: Autocomplete Isn't Understanding](https://www.forbes.com/sites/lutzfinger/2026/01/16/healthcares-ai-lesson-autocomplete-isnt-understanding/) · _Forbes_

Question 25

Why does Ben say agriculture uses more technology than early stage venture capital?

Accepted Answer

_Early stage VC relies on gut instinct without data validation._

At the early stages, VCs don't tend to use any data or technology. Growth stage is mostly a signal and projections problem, basically science at this point. But early stage venture is still very much a craft, flying blind mostly on gut instinct and what people have learned over time without really even having the datasets to figure out if what they're doing is true or not. VCs are starting to use Claude or ChatGPT but mostly to reproduce whatever they're doing. It's a sustaining innovation. They want Claude to evaluate a company like they've been evaluating them without asking if they're even evaluating them right.

— [The Hidden Patterns, with Ben Orthlieb](https://www.lutzfinger.com/transcript/2026-04-23-the-hidden-patterns-with-ben-orthlieb/) · _The Keynote on AI_

Question 26

Why does Clubhouse's future success depend on using user data?

Accepted Answer

_Finding content on Clubhouse is hard without better data use._

Clubhouse's future success depends on how it can help users find content and enable engagement through the use of user data. At the core, any social app solves two issues: to find and enable social interactions. Currently, finding interesting conversations on Clubhouse is not easy. You curate your own content by following people, similar to how Twitter used the social graph as a first approximation for your interest. A social graph is a clunky way to curate your feed. In order to scale, Clubhouse will need data and engagement metrics.

— [Clubhouse's Future Depends On Data - How To Build A TikTok Like Algorithm](https://www.forbes.com/sites/lutzfinger/2021/02/09/clubhouses-future-depends-on-datahow-to-build-a-tiktok-like-algorithm/) · _Forbes_

Question 27

Why is eval data a moat?

Accepted Answer

_Evaluation data as the strategic differentiator._

> All of them matter, and all of them are being fought over. But the real value may sit in the one thing OpenAI, Anthropic, and Gemini have not yet made central: your own eval data. Sure, there is the occasional thumbs-up button. But who is really using that signal?

— [The Missing Moat In AI: Your Eval Data](https://www.forbes.com/sites/lutzfinger/2026/05/26/the-missing-moat-in-ai-your-eval-data/) · _Forbes_

Question 28

Why is measuring social media so difficult despite all the tools available today?

Accepted Answer

_Tools are everywhere, but strategy comes first, not scores._

The problem is that measurement used to require specialists with big databases and tooling. Now with social media, you suddenly have all these tools that give you cloud scores and metrics instantly. People see a score and decide something based on it without planning what they actually want to achieve. If you haven't formulated your business goal first, any measurement won't work. People start with the measurement and then think about what to do with it, while the correct way is the opposite: start with a question, then come up with a measure.

— [Measuring Social Media What Marketers Need To Know](https://www.lutzfinger.com/transcript/2014-08-22-measuring-social-media-what-marketers-need-to-know/) · _Social Media Examiner_

Question 29

Why weren't traditional banks serving the underserved population before Varo?

Accepted Answer

_Economic incentives and outdated systems excluded millions from banking._

There are two main reasons, focusing on economic incentives. It was really difficult for banks to make money off consumers with low-value, high-frequency transactions. Banks perceived them as higher credit risk because they used backward-looking systems instead of monitoring real-time cash flow. Vulnerable populations are also more susceptible to fraud and scams, creating high costs for dispute processing. The economic incentives made it very difficult for incumbent institutions to effectively serve this large population. Technology was needed to solve this.

— [Disrupting Traditional Banking, with Colin Walsh](https://www.lutzfinger.com/transcript/2025-05-22-disrupting-traditional-banking-with-colin-walsh/) · _The Keynote on AI_

Q&A, Analytics

Did you find other factors that influenced vaccination rates besides trust?

How did LinkedIn use knowledge graphs to solve the job title problem?

How do you measure quality in healthcare data?

How do you spot social media bots?

How does Marpai Connect use AI to overcome the small numbers problem?

How does Richard distinguish between jobs that will be automated by AI and those that won't?

How does TikTok's algorithm differ from Clubhouse's approach to content discovery?

What are the three student personas you identified through data validation in peer reviews?

What did Ben learn at LinkedIn about predicting company success with data?

What did Lutz warn about China's AI strategy back in 2019, and why?

What did the data show about high trust versus low trust countries?

What is a data product?

What is a knowledge graph and how does it differ from a taxonomy?

What is the 'Ask, Measure, Learn' framework?

What is the book Ask, Measure, Learn about?

What is the difference between virality and contagiousness?

What is the potential financial impact of addressing Type 2 Diabetes in self-insured populations?

What lesson did you learn from private equity's data acquisition spree in the mid-2010s?

What metrics do you use for data validation in peer review quality assurance?

What was Ultimate AI's key insight that made customer onboarding faster than competitors?

What would Clubhouse need to do to understand conversations and improve content discovery?

Who does Lutz expect will be Clubhouse's fiercest competitors and why?

Why did you choose e-commerce to study generative AI adoption rather than other industries?

Why did you mention Google's 2008 prediction capabilities and WIRED's 'end of theory' article?

Why does Ben say agriculture uses more technology than early stage venture capital?

Why does Clubhouse's future success depend on using user data?

Why is eval data a moat?

Why is measuring social media so difficult despite all the tools available today?

Why weren't traditional banks serving the underserved population before Varo?