Skip to main content

Mixture of Memory Experts (MoME) | Data Brew | Episode 36

Summary

  • Fine-tuning for accuracy: Sharon Zhou from Lamini AI explores proprietary fine-tuning techniques including memory tuning to improve model accuracy and reliability in production agentic workflows.
  • Tackling hallucinations: The episode covers deterministic and probabilistic approaches to handling unstructured data, mitigating hallucinations and building evaluation frameworks for robust AI systems.
  • Production lessons: Learn from GitHub Copilot's rollout and other deployment experiences about challenges in scaling AI applications and ensuring consistent performance in real-world scenarios.

Mixture of Memory Experts (MoME) | Data Brew | Episode 36

Watch: Mixture of Memory Experts (MoME) | Data Brew | Episode 36
In this episode, Sharon Zhou, Co-Founder and CEO of Lamini AI, shares her expertise in the world of AI, focusing on fine-tuning models for improved performance and reliability.
Highlights include:

  • The integration of determinism and probabilism for handling unstructured data and user queries effectively.
  • Proprietary techniques like memory tuning and robust evaluation frameworks to mitigate model inaccuracies and hallucinations.
  • Lessons learned from deploying AI applications, including insights from GitHub Copilot’s rollout.

Connect with Sharon Zhou and Lamini:

Chapters

FAQs

What is memory tuning?

Memory tuning is a proprietary technique that optimizes how fine-tuned models retain and apply learned information, improving accuracy on specific domains without hallucinations.

How does fine-tuning improve AI reliability?

Fine-tuning adapts general models to specific domains with domain-specific data, dramatically reducing hallucinations and improving accuracy for production applications.

What challenges arise when deploying AI?

The video discusses lessons from GitHub Copilot including managing user expectations, handling edge cases and maintaining consistent performance across diverse use cases.

How are agentic workflows different?

Agentic systems require higher accuracy and reliability since they make autonomous decisions. Fine-tuning and memory tuning are critical for ensuring these systems behave consistently.

Full transcript

0:00 Introduction to Data Brew

Welcome to data Brew by data bricks uh with and the series allows us to explore various topics in the data and AI community and whether we're talking about data engineering or data science we will interview subject matter experts to dive deeper into these topics while we're enjoying our morning Brew I'm a principal develop advocate here at data bricks and one half of data brew and hi my name is BR wenig I'm the director of our machine learning practice and the other half of datab bre and today I have the pleasure of introducing co-founder and CEO of Lanai and many of you probably know from many of her online courses including courses on Gans diffusion models fine-tuning and improving the ACT receive llm applications so we are thrilled to have with us today for having me awesome so to kick it off like you're a public figure people know you from your online courses but what is lini okay so lini was more BN from a gap that I'm seeing in the market um and the Gap is you know enabling everyone to build their own Super intelligence because I I find this technology extremely magical generative models generative Ai and um it's enabling um every business to do this and giving them that infrastructure to steer language models to high accuracy on their own data and basically giving that infrastructure that makes it easy for these companies to build highly accurate Ai and I I think accuracy is honestly what's missing for Reliable agentic workflows today which is where a lot of um these workflows are going um and this you know infrastructure includes both proprietary data pipelines as well as proprietary fine tuning methods um including U memory tuning is something that we uh recently have put out as well that's really cool and I guess one of the things that I'm curious about is as you're building your company what what are you currently trying to address right now because just like has sort of intimated you're doing lots of different things and you're out out and about talking a lot of different topics about everything you just mentioned so just what is like what's the current Focus for you guys right now yeah the main focus is honestly this accuracy piece like getting people so that other data science scenes where you don't have to be deep experts with phds like myself um or folks on our team you don't have to be open AI to be able to leverage uh the power of something like fine-tuning um or creating data pipelines that are um that powerful as well so that the AI can actually do what you want it to do um and I yeah and I see this as again the major Gap in the market of uh between the amazing general purpose models that we have today that can maybe read Financial earning reports also has to read uh read your kid bed time bedtime stories right so they have to do all these different tasks but they're not super reliable um on on your specific tasks across all of your data hitting specific apis um being able to do text to SQL effectively um to be able to um yeah be able to function call effectively within an agent um so that you can actually rely on it so it can actually be automation okay sorry uh so the one thing you as you noted it's all about accuracy so I guess there's an implication here for which we even talked a little bit off um on the background here is that basically hallucinations suck right and this is one of the real key context in terms of where is is that real thing about accuracy or is there anything else that you really want to like dive into here I think that's a huge huge piece of it so um you know there have been a lot of Articles including those coming out from open AI saying that these language models are designed to hallucinate right um it's it's a feature not a bug or um uh I think there's a a fun article called jbd bullshits I'll put that in quotes um and so how do we get how do we get these models to not do that and also why why do they do it and a huge reason why is that they're optimizing for what's known as generalization error or average error across all examples over the internet and as a result they're pretty good at everything but they're perfect at nothing and it turns out that experts like yourselves like me like other people we are pretty good at everything but we're actually quite perfect at certain things I know I know Perfection is maybe overrated but we actually are perfect I remember your name you remember mine you can remember a bunch of you can grab the right you know um Revenue number from an earning report for example and not hallucinate on that um when you take a base um uh llama or chat GPT or some type of model they will actually hallucinate on Wikipedia pages that they clearly seen in their pre-training data um and we've even seen cases where it'll uh get 30% accurate if you ask about facts inside of like the Golden Gate Bridge Wikipedia page for example which is appalling in a sense um but it's also designed not to um not to necessarily be highly accurate on on on those fact and so as a result um they don't actually these models don't actually have that uh kind of internalize in the weights um and to make that possible uh requires um an extra piece of effort and that's what we really focus on the infrastructure that makes it so that it can remember all those facts in the Golden Gate Bridge it can um understand millions of your products in your catalog it can um and get the right ID right and put an item in the user's shopping cart effectively it can um write C equal to um thousands of your tables um and join them appropriately so it can actually do those things reliably with nines of accuracy and so throughout these various explanations you've mentioned sequel to sequel a few times oh yeah and you guys like that's why I'm kidding all right is to stop bringing up my past here my goodness I'm sorry any type of interface to any structured unstructured merger Lakehouse situation anything without snow okay sorry anyways let's one so I'm just curious what are the most common use cases that you see yeah such a good question so I think the umbrella is around agentic workflows quite frankly I think that's where a lot of people are are thinking about things just to you know frame what that means I think one really awesome analogy that a customer um kind of came up with when I was talking to them uh was an agent really is uh kind of a robot a program with a brain and llm or slm inside of it um and I really I really like that framing because I think that does make sense cuz we had agents previously right AI agents but the brain wasn't as smart and now the brain is much more intelligent um and some of the techniques the infrastructure that we built is making the brain more intelligent to develop expertise as well as memory okay so um so for agend workflows underneath um those use cases would be Texas SQL is definitely one of them being able to interface with any type of external system whether that be database or CRM um or or otherwise any kind of internal system um it could be a third party tool right um so so so those types of things there um I think another type of use case um Beyond code uh different types of code um are around function calling um and U making you know tool selection tool use planning so being able to actually um be very reliable there we have a um company uh copy AI with AI first company um is a customer of ours um that's classifying across thousands of classes over thousand classes and uh they have three nines of accuracy so like 99.

6:33 Agentic workflows and use cases

9% accurate to be able to that uh classification and some of those classes can be like functions ET so function calling um and some not uh so uh so I think that's pretty compelling so that you can actually rely on it you're not actually keeping humans in the loop and just shifting human labor in different places uh and then and then yeah um Beyond uh beyond that I think um there's uh you know there's also uh you know a bit more you can do with document reasoning um as we kind of push that envelope uh so if if you're just doing search you should definitely use retrieval augmented generation if there's a snippet in there that you want the model to reason about but if you want the model to reason across documents um basically develop a new capability then you probably do need to tune it as well so um that's probably the final area so one of the things I'm curious about so we've talked about these different use cases and that you're seeing here right so we You' talked about Texas SQL you talked about function calling right but uh within the context of these agentic workflows where there's a as as you described a brain that makes the agent a little bit smarter but is it is that all it is is just basically putting an LM and slm and you're good to go or what's like the magic behind it to make these brains or these agents actually work better than the what they did in the past so I I think a huge part of it is um one obviously the probably the biggest lift is um tuning it so that it actually can have this expertise in memory exactly how we do it is a type of tuning called memory tuning um which is a high bandwidth High Fidelity mixture of experts uh lauras on top of a pre-trained model um so just it is uh basically tuning a lot of adapters on top of a model uh that you know are routed to at the right time um given in query uh so this is a highly effective way to get the model to essentially internally retrieve right so using information retrieval just the same parent as rag um to internally retri r the right um weights uh to supplement its reasoning um so that's that's effectively what uh the technique that we're we're using to make this possible Sor did I answer your question I forgot your question no no you did actually because you definitely gave the the mathematics and the concepts behind it right which is to explain like basically this brain's a heck of a lot smarter than what it used to be before and then and you're using these particular techniques is this like the only way by the way to do it or do have you been researching like all the different other techniques as well like I'm just curious because from your perspective I'm sure there's it's one of those cases where sometimes this will work great and sometimes not so much right oh yeah absolutely so um I think this is a coupling of two different techniques and taking them to the extreme one is um so was yeah very um uh famous and influential paper in the space uh basically uh teaching external weights outside of the model on top of the model exter weights on top of the model um so that you can swap in and out a a pre-train model underneath and it's just a very efficient way to tune the model without losing accuracy so instead of tuning the entire model um know it's just tuning Parts on top um and uh you know you can get up to I think 10,000x more efficient on a gbd3 level model which is huge like that is in a huge efficiency Improvement um and I think the whole industry has kind of moved towards this direction uh so that's that's one I think um second technology uh that has been essentially taken we've taken to the extreme is mixture of experts uh so mixture of experts uh gb4 famously mix Etc there a bunch of model pre-train models out there that are mixture of experts usually eight way for stability you know from that uh standpoint for training um but essentially it's you can think of it as um using only part of your brain not your whole brain to answer a question um into route to only part of the brain which means lower inference cost and lower inference latency right so um so being being able to do that is is uh important um for yeah for um keeping uh keeping the thinking part as fast as possible um and in specializing it um so now if you marry these two together um you uh get kind of what we've uh created which is a mixture of memory experts or we call it mommy yes that's right and mommy knows best we as they say um so it's a mixture of memory experts M and I love this tagline oh that was almost a paper title but yeah there's a little bit of veto you should have why not oh I could sorry we weren't spicy enough um we should have you're right uh yeah come on I expect I expect better from you Shar I I know you do and I will I will try for the next one so goes F but mommy um uh the mommy model um is a mixture of millions of luras on top of the model right that are select that's run time yeah to dynamically put together a sub team of them to answer a question um so that the Right facts are kind of compiled and it's it's huge right it's massive it's it's a highly parallelized process um and then fused back into the model and all like makes it extremely efficient to run uh run inference on and of course also benefited from thee architecture so that's that's what we've produced um and in terms of and that came from working with customers like that did not come from just us developing random algorithms um in a in a lab setting this is oh this was clearly an issue this is one way to tackle that issue in an extremely um compute efficient way um so so that's how we that's how we invent to so I know a lot of customers want to do fine tuning and some of them are a bit hesitant of like hey should I just wait for the next like iteration of llama to come out uh cuz that might just solve my problem other ones know like llama's never going to see my proprietary data therefore in your words like it can't be excellent at it and so just curious how you balance that trade off of should I invest in uh fine-tuning is it really worth that performance benefit yeah so I think two things one um it actually is less costly it's more cost ineffective than a vanilla model if you do the right thing so if you distill it it's a much smaller model also if you don't distill it um it's a much smaller prompt right so um so for and for given accuracy threshold that Paro Frontier it's actually much better um and you do have to run the model sufficiently you can't just be like oh I pinged it three times then you should definitely use a general purpose model uh but if you're actually getting high utilization on it if you're pinging it thousands of times like in a production Enterprise setting typically then you do want um to you know then at scale the costs do actually make sense so that's that's one thing on cost um though I don't think people are optimizing that super super heavily now it's in the back people people's minds of course but it's not being highly optimized yet um I think uh what was the first part of your question um oh people are thinking about find Jing kind of Dipping their toe in a little bit scared um I I think the right I let me just walk through kind of a maturity curve where I'm seeing people do things so I think at the very um start of the maturity curve is exploring you know I think this is really 2023 of um chat tpd just came out two months ago and I don't know what this magical technology is can it do everything can it do nothing what where are the boundaries of it so just really exploring and touching this this this new type of Technology um and doing basic prompting uh so that's one and then uh the next step is um I think where people are experimenting a little bit more and they are applying basic Rags so like connecting their external data they're doing Advanced prompting um so they under you know they understand this uh technology a bit more they're wetting it with um information retrieval information retrieval plus AI so that's that's really powerful um to kind of get a little bit more from these models um and then the third is uh around implementing so I think like people Implement starting to get uh use cases in production which is what I'm seeing this year so like Enterprises are putting use cases in production um however a lot of them are what they call shallow and they come up to me and say is this it is this really it you know is this really the magic of AI because honestly I was expecting more and I'm like be a little patient this is the most you can get out of uh doing weekend projects so if you invest a little bit effort one can see more Roi so I think that's a function of that and people are of course like learn there's a big learning curve here so um that I think that's where people are I think uh starting to touch fine tuning obviously you know a little bit scared in some ways because um it is harder than just hitting an API right it's actually uh actually I don't think people fully realize this it's it's tied to the data Pipeline and evaluation in a very very coupled and I know I mentioned this all my classes but it's it's so coupled people don't understand this except for experts um that like if you look at openi their most proprietary thing is probably their data pipeline evaluation like it's probably not the tuning piece even exactly that piece right they're not exactly that moment of backrop it's actually the cycle of this whole flywheel that they have so um I think that getting that right is is tough and so we're we're working on ways to make this easier we've already you know put out some of these things mem tuning is is is a method to make that possible Right but it is still um uh I think more difficult quite frankly than than doing basic rack um because the Cycles are longer for iteration and it does um it's closer a little bit to what data scientists do as opposed to just developers who I know are playing quite extensively with these models and so that's that's what we're that's what I'm seeing and of course I think um on the mature side of things um people who are for sure those who are innovating on in the space or just ahead of the curve they will um they will be fine-tuning more comfortably um uh and they will have mature data pipelines and evaluation uh metric set up so yeah uh one of the things I'm curious about is you you talk about data pipelines which of course now is much closer to my wheelhouse right and okay one of the things I'm curious about is like do you do you feel whether it's the ETL prolite itself or the generation of synthetic data do you find that these systems are improving the flow to get to help Enterprises or companies go ahead and start doing fine-tuning or is it just about like no just organize your data better and then you're good to go like like do you what do you find um is there like a I guess almost like a Magic Bullet per se that's helping people to build better pipeline or it's just really it's a it's a natural problem uh based on the type of data they have uh my view of it is uh we're actually in a special time where the organization of your data doesn't matter as much anymore um because we have models that can help you process that now you do have to understand what you're doing with the models and what they're capable of uh and what they're not and so I think and also what will help them improve themselves right so you're creating a data pipeline that will improve the model itself so it's a self-improving model which is magical when you try to extrapolate it but um in in the in the short like first couple iterations it might feel a little painful because as you it it's similar to prompting like prompting maybe you said the wrong thing and you're iterating but it's e that's easy because you're just editing text and you're getting response back really quickly within seconds right um I think for for this case your iteration cycles look more like hours at Best and um and you're editing a whole data pipeline where you have to understand the model at the same time and I I don't see that a lot right so I see people who are excellent dat and I see people excellent AI people and then it's kind of like oh actually if you're excellent at both um this would be very easy but otherwise this is this is actually quite Challen can be quite challenging the first couple times um but I am seeing uh methods to automate this more and more I mean we're definitely working on that because I think that's where some of the biggest pain points are now what's challenging is I do think there's a level of customization across every single use case and across every single customer um and that part of that is quite frankly um uh part of the customer's Mo like that will be part just like it's opening so if you're building so like it's this weird you know like oh it is part of your it would be part of your AI in a sense so um it is important to kind of understand what's going on but the interface to building these can be much much easier and there could be automation which we're definitely working on uh to make that easier um an easier lift for more people to touch um rather than than those who actually understand both both sides right so in other words you're getting to a point where just like you had mentioned earlier right you don't have to necess have a PhD in the space in order to be able to to actually automate it like that you can we can build these tools to actually in essence make that magic simpler to build these pipelines and just as long you actually can the Two Worlds basically together yeah yeah like why should only anthropics set the definition of safety in these models right why can't like there should be like everyone should be able to set their own and it's it actually is different across different organizations it's not like they're passing some moral boundary they're actually they sometimes it's just like a real business case to have a different boundary so um yeah so at the beginning at the high level you mentioned like the biggest challenge is strugling with accuracy where would you say are the biggest areas for improvement is it that people don't have accurate evals is it that they don't have the data to do fine tuning just curious give the variety of customers that you work with like what are the biggest areas for improvement yeah so I think uh evals is a big one I see Bad Evil set up all the time people using LM as judge incorrectly um and they're like this is 70% correct and I'm like well when you look at it it's 20% correct if you actually go judge the judge can you can you actually talk about that a bit more like what what is a bad element as a judge well I think when you rely on it Without Really checking it um but quite frankly I think that's also a symptom of for my view that you don't care about accuracy that much does that make sense or you haven't even put in the right systems to care about it so if you're evaluating it in a you know loose way you clearly don't care about it that much um and uh I I kind of started to think about how to find the you know we mentioned agentic workflows but like I think the class of use cases that where accuracy matters the most are those where um failure is both clear and consequential and I think that's why um for example that's why Texas SQL is interesting because you generate SQL it fails to hit the database and give you the right answer like it just there's so many clear like in software fatal errors right that is so clear and consequence is so clear so it's just so clear and I think in that case you know you can get a benchmark where it's it's just clearer it's not disputable or it's not like requires a human to not be lazy now in a factual kind of assistant QA type thing um do you want to go verify that this llm is correct or not uh reading through all these insurance documents right like do you and then when you look at it like that's where we see people have errors on errors on errors and they're like oh no but it it's consequential failure is consequential because we're going to get sued and I was like you're going to get sued but it's not clear the problem is it's consequential but not clear so it has to be both um yeah so that's that's what I've found that's how I've kind of distilled distill that boundary of like what use cases and of course in agentic workflows a huge issue is um tool use and selection you're basically hitting URLs right like and if it hallucinates URLs with just like hitting a function with you know parameters like it's not going to work if you hallucinate the wrong URL like if I hallucinate wrong URL I'm not going to show up in this podcast so it's clear and it's conse quential um in that sense and I think when you string along a lot of these because in an agenic Warf you got a lot of Agents kind of hanging out and interacting with each other um it's going to really show up it's going to compound so I think that's where uh in the demos it's it's really promising right because it's cool you show One path that works but the problem is these models are highly probabilistic and you want to be able to inject determinism in them when there needs to be deter determinism and you want there to be prob you want it to be probalistic because it's that's part of the magic of them as well to handle all that un structure data and to handle user queries um Etc so let's figure out the right balance no you know what when when you mentioned that you literally reminded me of in terms of like do you really care about accuracy it goes back to so one of my past lives was uh like was medically based right so oh uh doing uh medical research and it was all about and basically it was all about the P value right and so basically the idea oh P this we're good to go and everything's done and we've proven it and then of course you you scratch the surface just a little tiny bit you're like your hypothesis not even close to correct what in God's name are you talking about right so it seems like to your point exactly right to your point exactly it's like yeah great you've got a number like an El judge says 70% but to your point exactly he's like do you did you even build up a structure in the first place to actually do the evaluation correctly nope all well then there's a problem so yeah yeah that just me that belly belly XKCD yeah right we it's it's the it's back to lies damn lies and statistics all over again like we're back we're literally back to that again all over so exactly but it's telling when you when you don't double check everything triple check everything um but yeah I do find it kind of funny that is right now the a popular method and of course it's possible they do care about accuracy it's just that um they're not at that they're at a stage of development where they haven't interact with their business unit that is then going to be like actually no we're not putting this in production um but it is it is uh is it is uh it is a bit of a problem I think um yeah but maybe it's a symptom and so what are some of the biggest blockers you see from customers taking these use cases and putting them live in production like do you see different levels of scrutiny for internal versus external facing applications curious if it's more of like a technical challenge or more of like a organizational challenge oh I think a lot of it is organizational I mean there is a technical challenge right to serving so we we do have like this uh we make it possible to have a self-managed self-hosted kind of solution on both tuning and inference Etc like the whole stock uh orchestration everything on top of the GBU so all the stuff were on top of the GPU to run the llm so I think like there that is you know better to use something where it is best practices than cobbling together all this stuff um that might not be optimized for it uh so I do think that is a piece of it now do I actually despite the fact that I do sell that I um quite frankly from what I'm seeing uh I think the organizational structure is a is a bigger blocker for many of these organizations um and I think in some cases it rightfully is so in the sense that um internal use cases will obviously have a lower bar to uh shipping them in production um but external use cases I think I've seen some smart ways of doing it as well so like it's higher value typically right so um cuz it's you know they're getting more revenue from it Etc it's less so cutting bottom line and so I think uh what I'm seeing is ways that llms can support that workflow and automate parts of it but not be um but but quarantine when it comes to risk of it you know generating a super creative output um so I think uh there are ways to kind of mitigate that risk it can prod it can call functions it can call certain messages to send um but it cannot invented messages like chachie PT does right so it's it's something in between where I think it's a safer path to production and of course just like GitHub co-pilot um a lot of these things are in phased uh kind of release so I think like GitHub co-pilot I view as an Enterprise llm and an Enterprise use case and like one of the first right so um it was a phased approach and uh organizationally I know for a fact that it was very difficult to ship this in production okay despite the fact that it was open Ai and Microsoft or maybe because of I don't know but like there like very hard to get these two teams to work together and shiping and production in the right way um and it's a lot there's a lot of infrastructure but there's also a lot of organizational Dynamics um to making sure it meets everyone's criteria uh so you know whether it's performance from an engineering standpoint of hey there's got to be low latency because otherwise no one's going to press tab like it's never going to show up you know the tab You Know Tab completes number can show up and then the ml team is like way but we made the model even bigger and slower but now it's smarter and then the engineering team's like I don't care I needed low latency and so just I think there are different definitions of good across and then business team probably say something else so like we're not making money from it so like it's it's all these different facets that need to come together and I think it takes a lot of organizational will and I think it honestly takes a lot of leadership um to pull it off to like just be like we're doing this it's happening um it's our first we're doing it yeah and we're going to ship it absolutely and in addition to everything you just mentioned one of the trends that I often see is everybody wants to do Ai and so in addition to leadership there needs to be a clear leader of like not three different people saying they own the Gen strategy for their given organization but having single strategy yeah exactly and so I'm I'm very curious like you come from Academia like you're co-founder and CEO of L what were some of the biggest challenges or learnings along the way as you embarked on co-founding uh a company being CEO well okay I'm actually a total weirdo before my PhD was in Industry actually so I actually um have attempted to start companies uh start an accelerator previously so that about 70 people did quit their jobs and start companies um and was a product manager at Google and a couple startups before this uh before my PhD um and I am a total weirdo because I may have told on PhD adviser who's lovely and ring um that I would uh drop out the first day my goal is to find a co-founder drop out Larry Sergey style uh and I wanted to build a technologically transformative company uh and the joke is I failed to drop out uh because I fell in love with jeda of models which he um initially told me had no commercial viability and honestly at the time it did appear that way uh but I just fell in love love with these models because I think they're magical creative and smarter than me um so I just find them Ultra M magical um and that's super weird for me because I'm a people person I never thought I'd fall in love with a p technology so kind of My Hope For The World Is give this magic to other people make it so that other people can um work with this magic too um like I can and so that's why I love teaching um it's why I also that that's what the product vision is Right make it possible for everyone to build their own Super intelligence so um in instead of just hog it to ourselves as AI experts so I think uh that's that's where we're at and yeah so I feel like one of the biggest challenges um man starting a company uh I mean first I think I was very fortunate because I think the timing worked out really well um uh potentially there was better timing maybe Mosaic good job Mosaic um but I think I think uh we started a a few a couple months before Chachi BT was launched um and it already felt like a very exciting moment and I was going to sell to midmarket right like large startups the like and then suddenly every Board of every Fortune 500 Global 2000 company says this is our number one priority in November 2022 right so oh my gosh this suddenly became so relevant and I couldn't have asked for more than that I don't think any anyone in I expected it to be at this level of relevance to the world and so um I think it's a really special time and really special piece of technology and I I hope you know in this moment that I can I can share it with as many people as possible so that's yeah that's kind of my founding Journey um and yeah I uh yeah I don't I can't Center on exact you know challenges I will say that there there is a lot that happens I think um one thing is just a personal thing figuring out what my Cadence is so I don't burn out um I think uh figuring out like what I need to be you know to have my own space to be a in addition to you know being part of this company and not get fully swallowed up constantly by it um you know maybe I watched a Disney movie this weekend maybe yeah I typic yeah we don't go that um so so it's just you know like I think finding the right the right um the right uh things to be continue to be uh revitalized and energetic is is important while sprinting because I I used to be competitive Sprinter so I love sprinting quite frankly um yeah um so lots of I know yeah lots of berries now so that's been fun berries boot camp uh but lots of things to be revitalized and making sure I'm I'm taking care of myself as well well that's super important and by the way if you ever want to join Brooker myself that you can just come to the dark side and join us in cycling as well by the way so we make this whole podcast just about cycling if you want to no no no no you you should have seen how bad I was in 2019 okay yeah yeah yeah on the road like in 2019 I was in bad shape let's just put it that way so I've and now I'm only on10th of speed and now I'm actually as fast as one10 of speed of now so I've improved quite a bit now so so you know there are things that you can do to help yourself right so yeah yeah yeah see I see at the Olympic yeah I'll hit there in about 50 Years it'll be fine it'll be fine it'll be fine so but by the way I I before I ask you the one one more question I did want to call out which is sort of you still got my brain a little befuddled here I I've failed to quit I failed to quit out of my PhD that one still is yeah like if my parents if they ever heard me say that they would disown me I'm just calling that out right now so no no no but then they'd be like oh wait you succeeded it's it's really yeah I mean it was my goal and I did fail at my goal um it it also right so you would think you would think I'd succeed you had a nice way of hedging of either you completed your goal of you drop out and you co-found a company or you get a PhD like either outcome is a good outcome I don't know maybe not the way I grew up yeah um my dad has a BC and my mom was like well I earn more than him so I think it's always I don't know exactly what the value is that's just been ingrained in me so I was a little fearful of getting the degree quite frankly um yeah not in yeah the most rashing moment no no I I actually could understand I was uh both my parents have phds and my dad was in statistics as well so of course like that was the whole point it was supposed to be Bean in me and then of course you typically Taiwanese parents I was supposed to be a doctor so of course I failed on everything so you know boom here we go I failed it all but at least I'm doing Tech so thank goodness I guess I guess I guess oh man let's get back in the topic at hand because obviously now we're starting to reveal a little too much trauma to each other right now in this podcast but I just I'm just curious where did you even come up with the name because for I'm hopefully I'm saying it right right L because I I've done this enough where I've screwed it up okay the name it's a really fun story so we may or may not have been called llama in stealth uh we were um because llm llama letters are in there uh obviously Mark Zuckerberg went on when he went on CNBC to announce you know even when it was a research paper we're like maybe it doesn't matter and then we were like okay zck went on CNBC and talked about llama so probably not our name the logo is already designed we I own like 10 to 20 domain names with the word llama llama company whatever in it um own the train Mark not in public use but they did contact us so anyways we're in that situation and I think okay well we need a new name and uh a lot of fun kind of stories of generating names uh the language models were definitely not good enough at generating good names then um I don't know if they are now but they definitely were not then um one time though chat GPT did say uh did Judge us and said why are you spending all this time generating trying to generate asking me to generate names you should be working at your company and wow so I think uh I know I know I I really it was a very judgmental moment from Tatu BT uh and ultimately one the founding Engineers actually um thought of the name because lini is from uh is actually the pseudo Latin term for the tribe that llamas and alpacas are a part of so if you check on Wikipedia that's actually the tribe it's not like the genus or like you know like like that genus family but tribe and so uh we're like oh that's perfect uh it's Unique um uh hopefully no one's going to take it um and then uh yeah I think I think it's relevant to the Llama theme um and one little add of benefit which was I did not anticipate was I didn't have to add that many letters to the logo but if you look closely you can tell which parts I designed and which parts were designed by the designer uh but yeah let's just leave that in there uh I by the way it's really hard to draw a vector n um so I think uh I I think so that's how we came up with it um and we went with it and I'm really happy that dead um yeah I really I love her name um maybe the one tricky thing of course is pronunciation people are like lini lamini um uh which is which is fine we we maybe should have gone with those but uh lini it's like llama plus Gemini so language models there we go I love how the name works on so many levels yes it does it absolutely does well we just want to say thank you so much for joining us stay on data Brew enlightening us on the success you can have from not always achieving your initial goal oh my God now right right mommy llamas yeah we have this like llama with little baby llamas mhm oh I I love how forward thinking you were with everything lava before it was officially released so we now have that on recording too it's fine unfortunately you the have it of public you do matter so so don't stay in South for too long would be my advice um oh there you go okay both boom final piece of advice for yeah founder advice in llm land just build out in the open Don't yeah don't hold back I think I've learned that lesson the hard way multiple times one of them is the name also learned that for not the but I think you're also going to take advantage of the Llama name too so I I think there's a lot of benefits that came from the approach that you took yeah well we just want to say thank you so much for being here we really enjoyed our conversation with you today and we hope all of our listeners did as well thanks so much as well thanks so much for having me

Learn more about the Databricks Data and AI platform.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.