What I learned from two days of hanging out with AI experts

Everyone has an opinion on AI. There's so much noise and hype around this technology that it can be truly hard to figure out what's really happening. So when one of the co-chairs of the Toronto Machine Learning Summit invited me to their IRL conference last week, I decided to find out how people who are living and breathing AI every day are thinking and feeling.

I'm still an AI pragmatist

As I've gone over in previous posts, I'm open to exploring and experimenting with AI but I feel pretty strongly that we need to actually measure whether adding AI to a workflow produces any time savings at all. There are some use cases (such as transcription, migrating frontend styles from design programs like Figma, and summarizing code or content) where AI is measurably fast enough to actually still save time even when you add in the time needed to fix its errors. There are other cases where AI either makes you waste more time from fixing errors or the amount of time you save doesn't come close to being worth the cost of the tokens, much less the environmental and ethical costs that come with this technology as well.

One conference isn't going to stop me from being a pragmatist, but I was definitely curious how many like-minded people I would meet. Here's what I learned and observed.

1) The hypesters are outnumbered (at least in Toronto)

One of the most telling moments for me was when one of the hosts did a poll of the audience and asked with a 1 to 10 scale (where one is "not going to happen" and 10 is "definitely going to happen") whether artificial general intelligence (AGI) was going to happen within the next 5 years. The final score of the answers was around 2.9. That didn't surprise me given the number of researchers and PhDs in the room. AI companies that stand to make a lot of money off AI hype have a tendency to present artificial general intelligence (AGI) as imminent because it either scares people or gets them excited about investing. You ask the average machine learning expert when they think AGI will happen and very few of them will say it's happening in the next few years. The fact that the audience was dubious of the 5 year timeline was a good indication that the people in the room were focused on practicality instead of hype.

Most of the people I met were definitely more interested in facts than vibes. Also, this might a testament to the Toronto Machine Learning community more than anything else, but there was a huge amount of diversity both in the audience and on the stage. It's pretty refreshing to be at a tech conference where there's a line out the door at the women's bathroom in between talks as well as the men's room.

2) Being model agnostic is the way to be

A phrase I kept hearing over and over again was "use the right model for the right task." A lot of speakers and other experts I met are encouraging people to stop relying on a single model for everything they do. While the idea of a single assistant who can do everything for you might live on as people experiment more with OpenClaw and autonomous agents, it's unlikely that even those agents will rely on a single model for doing everything. Using an AI model optimized for coding to summarize emails is kind of like mixing gold dust into your coffee — it's not a good use of resources. But that's what a lot of people are doing right now. They're using one or two overpowered models for everything.

One graph that appeared in multiple talks was this one showing that the capabilities of cheaper open weight models like DeepSeek and Qwen are catching up to the frontier models. This is bad news for the giant-model-that-does-everything approach because a lot of the speakers were encouraging people to widen their horizons and experiment with a mix of open weight models to find the best models for particular tasks. This is a trend that will pick up speed because now we have a better idea what the true costs of the frontier models are and people are going to be hunting for cheaper options.

3) Return on investment isn't optional anymore

Speaking of money, value and return on investment (ROI) were also recurring themes. As one of the speakers said, "The technology is being asked to justify itself." And right now, ROI from AI isn't emerging for a lot of organizations. Many speakers spoke about strategies for getting more value out of AI models, such as matching model to task, exploring the performance of cheaper open weight models, and optimizing the context and code around your API requests to make tokens go further. One speaker even presented a value framework that showed how it can make more fiscal sense for organizations to invest in other solutions besides AI. It was clear from the way people were talking about AI performance that business leaders are growing tired of AI experiments and want to see results.

4) Sovereignty is a big concern

Along with the rising concerns over cost, another theme that came up over and over again was sovereignty. Multiple presentations mentioned building "sovereign AI" systems where organizations could either train their own in-house models or develop workflows to reduce switching costs if a particular model or AI company suddenly becomes inaccessible. The news of the United States government removing Anthropic's latest models from the market was pretty fresh at the end of last week, but the concerns about sovereignty were already there and aren't going to go away.

5) Bigger isn't always better

Local models that can be run on your machine are getting better and shinier. One of the coolest talks I saw at the whole event was by Helena Yu and Luis Ticas, who are part of a Toronto nonprofit called Sprout Climate. They built a chat bot that provides evidence-based answers to people's questions about climate change in over two hundred languages using a combination of small, local models rather than relying on massive frontier models. Right now, their chat bot only costs them CAD$26 a month because they chose to build with models that are more sustainable in terms of cost and environmental impact. This is a trend other smart people have observed and it's one that's likely going to grow.

Conclusions

Overall, it seems to me that the attendees at this conference were pretty representative of what Anil Dash calls the real majority AI view. There were some people who were more driven by hype than facts, but most people were there because they're working on creating legitimate uses for machine learning technology. I imagine attending an AI conference in San Francisco would be a very different experience.

Seeing these trends and seeing how machine learning experts talk about their technology makes me glad we chose the approach for Wagtail AI that we did. It seemed important at the time to give users as much agency and choice about AI models as possible, and it's vindicating to see the way we designed the package will help position our users to take advantage of open weight models and local models if they want to.

There's still a lot of noise about AI out there. But if you're fortunate to spend IRL time with experts (not the "experts" on YouTube or TikTok), then you'll find there are a lot of earnest, thoughtful people actually trying to solve problems who are getting drowned out by mindless cheerleaders and fearmongers aiming to please their investors. While there are still a lot of issues surrounding this technology and it's going to be harder than it should be to reign in the most irresponsible AI practices, seeing what the people at the Toronto Machine Learning Summit were talking about gives me some hope that more practical and ethical versions of this technology could emerge from the hype.