dagelf · April 1, 2023 23:49 · dagelf · Apr 2, 2023
diff --git a/gistfile1.txt b/gistfile1.txt
 Source: https://twitter.com/i/spaces/1jMJgLdenVjxL
 With timestamps: https://gist.github.com/dagelf/b94763167c5042181707479f4c815f9e

  Let's see, hello everyone.
  Hey everyone, David Shapiro here.
  All righty, hello everyone, we're going to give folks a few more minutes to join.
  We're also waiting on Elon.
  So we'll get started shortly.
  Let's see, hello everyone.
  Can you guys hear me okay?
  Loud and clear.
  Okay great.
  Well, we're excited to open source the Twitter recommendation algorithm.
  This is not everything, but it's the vast majority of it, and there's a few parts that
  we need to just do some security and privacy checks on before we open source it.
  But the intent is that we open source everything that's necessary to predict with reasonable
  accuracy what will be recommended to people on Twitter.
  So the idea here is that why should you trust any social media company or any company if
  the recommendation is a black box and is subject to manipulation in ways that you
  don't understand.
  And it could be from people, from just code errors, it could be, you know, it could be
  state actors potentially.
  Anyway, the point is that you need to be able to trust what is being recommended to you
  that you at least understand where the manipulation is coming from.
  And our overall goal with this platform is to achieve maximum trust, and maximum trust
  comes with maximum transparency.
  In other words, it can't just be us saying, yeah, we're doing the right thing, but when
  it comes to software you can actually see if we're doing the right thing.
  Now, what we're open sourcing today, to be totally frank, is it's going to be pretty
  embarrassing because there will be lots of mistakes that are found and lots of improvements
  that we'll make.
  And of course, some people will try to exploit probably what they see as flaws in the algorithm.
  So I would expect like the next few days to be somewhat bumpy as various people and organizations
  try to exploit the algorithm.
  But we will then identify those weaknesses and fix them.
  And you know, there is this whole debate of like, well, should, is it a good idea to open
  source something like this?
  Because what about manipulation?
  What about, you know, smart people and organizations using the algorithm in a way that's bad?
  And it's like, well, okay, but if we open source it, then the whole community gets to
  look at it and gets to identify issues to get fixed.
  So it can't be manipulated, or at least it is the least subject to being manipulated.
  It's not like the Twitter algorithm right now isn't being manipulated, it is, by people
  reverse engineering how it's working and then essentially gaming the system in hidden ways
  that other people aren't aware of.
  So but by open sourcing it, the whole community gets to look at it, the whole community gets
  to comment and make it better.
  And I think in the end, it will result in the most trustworthy, most robust algorithm.
  And I think it will be the best outcome.
  And you know, I think the best example for open source is Linux, and you think about
  Linux, if you have a complex operating system, again, you know, a lot of people attack the
  open source nature of Linux and said, well, if you can see how the operating system works,
  you can figure out, you know, packing exploits and figure out how to, you know, gain access
  to sensitive information, because you can see all the details of the how Linux works.
  And but that actually turns out not to be true, because people, the community identifies
  these issues and fixes them, and ends up actually Linux as an open source system, being more
  robust than closed source systems.
  And that's the that's the intent here with the Twitter open source algorithm.
  So with that, we can, there may be actually, I should say, others on the Twitter team who
  may wish to say a few things.
  Yeah, thanks, Ilan.
  So we've got some of our engineers on the call today, who were part of this project
  and who have built many of these systems.
  We'll start by kicking it off to Tejas, so why don't you take it from there?
  Hi, I hope everyone can hear me.
  Hi, I'm Tejas, engineer on the Twitter team.
  And today, I'm super excited to share that we're releasing set of services and jobs that
  are responsible for constructing and serving the tweets on your home timeline.
  This includes various tweet candidate source features, including embeddings used to find
  the best tweets for you, machine learning model architecture features and model serving,
  code to construct the timeline of tweet, how it's constructed.
  You can find the detailed overview of our algorithm in the in the engineering blog that
  we shared earlier from the Twitter edge, a handle and entire code is live on GitHub right
  now. And now we'll be going, dabbling into different parts of the system that we would
  like to talk in detail today.
  And I'll be handing over to Hafe to give the overview.
  Hi, everyone.
  Can everyone hear me all right?
  Happy. Yeah.
  Hey, I'm healthy. I'm very excited to be here today.
  I'm an engineer at Twitter.
  As Tejas mentioned, Twitter's recommendation are built to answer important questions about
  the Twitter network, such as what is the probability you will be interact with another
  person in the future or what are the communities on Twitter and trending tweets within
  them? Answering these questions accurately enables Twitter to deliver more
  relevant and compelling recommendations.
  The recommendation pipeline is made up of three major stages.
  First, fetch the best tweets from different recommendation sources in a process called
  candidate sourcing. There are two major types of candidate source in network and
  auto network. In network tweets come from people you follow, auto network tweets
  coming from people you don't follow.
  Then we moved on the second stage, scoring, scoring the fetch tweets from the previous
  stage using a machine learning model.
  And then moving to the third stage, we apply product product features, constructing the
  timeline and deliver to you.
  We will dive into each stages today.
  First, Brian, do you want to talk about in-network Twitter generation?
  Yeah, sure. I hope everyone can hear me all right.
  So, yeah. Hey, folks, my name is Brian.
  I'm an engineer on the timelines team working on retrieval of tweets, specifically here
  in network or from tweets from people you follow.
  You can think of this as pretty much the largest candidate source that we use to deliver
  the most relevant recent tweets on your timeline, since these are people you have
  explicitly chosen to follow and you want to see their tweets.
  At a high level, it's a pretty simple system.
  We find tweets from the people you follow in the past 24 hours.
  We rank them by a relevance function, and then we take the top tweets to return to the
  next stage of the timeline creation.
  Maybe the most important component that I'll highlight here in ranking of the in-network
  tweets is what we call the real graph.
  You can think of that as a model which predicts the likelihood that you're going to
  engage with any user that you follow, pretty much based on how many tweets you have
  pretty much based on how much you've interacted with them in the past.
  The higher the real graph score here, the more of their tweets will include in your
  timeline. And that's about it at a high level.
  This is probably our most straightforward component of the for you algorithm.
  So next, we should go over the high level of how we determine which tweets to show you
  from people you don't follow, which we call out of network.
  So, Eli, do you want to take a stab at it?
  Sure. Thank you, Brian.
  Can everyone hear me clearly?
  Cool. OK. Hi, everyone.
  I'm Eli. I'm an engineer in the timeline teams who is working on the auto network
  recommendation problem for a long time.
  Auto network recommendation actually is very interesting and very hard work.
  So we right now we have two kinds of major recommendation system.
  What the first one is called the social graph based kind of generations.
  We try to find the tweets, which people you are trying to engage with in the past few
  past hours. So, for example, if your friends are talking about the earthquake
  happens in the surface, it's probably this week.
  So maybe you are receiving the kind of prediction about the earthquake.
  Another kind of trigger conditions is based on the code embedding based or embedding
  is a word we use in the machine learning word for for for other use cases.
  It can be understand as the representation of the entity in a certain space.
  In our two recommendation systems, we found a similar tweet with the stuff you
  already part engaged with before.
  So, for example, if you like some NBA tweets, we will definitely give you more NBA
  tweets based on the kind of embedding space.
  We also build our user embeddings to represent your long term and short term
  interests. Right now, we have about two major kind of tweet embedding spaces in
  our production. One is called sim clusters, which is sparse embedding.
  And another one is called tween, which is dense embedding.
  We have all published papers talk about how we build these two embeddings system.
  And feel free to check our blog.
  You can find them and all we have all the codes already available to everyone to
  check in our GitHub report.
  So both of the sim cluster embedding and the tween embedding are also being used in
  as a machine learning feature in the ranking stage.
  So, JJ, would you like to give us more details about how our ranking works?
  Yeah, thanks. Hi, I'm JJ.
  I'm an engineer at Twitter.
  I relatively recently started working with the timelines team.
  And yeah, so what we've done over up to now is how we get like a set of candidates.
  So that's like a fairly large number of tweets that we can potentially show in
  your timeline. And then we use machine learning models to try and determine which
  of these tweets are likely to be the most relevant for you.
  So the way this works is we use thousands of features as input to these models.
  For example, have you engaged with a another tweet by this author recently would be
  one feature we use as well as some of the embedding features Eli just mentioned in
  the code that we open source, we have quite a detailed outline of a detailed
  explanation of the features we use.
  And then the model makes using this input makes some predictions about, for example,
  are you likely to retweet, like or reply to this tweet, among other things?
  And then so we combine those predictions to arrive at a score of how relevant we
  think this tweet is likely to be to you.
  And again, the full details of exactly how we combine that scores in the code we've
  released. And so then we take these scores to determine which of the candidate tweets
  might be the most relevant for you and should show up on your timeline.
  And the mechanism for that, Ishan will go over next.
  Hi, everyone. I'm Ishan.
  I'm an engineer here at Twitter and I work on the backend systems that serve and
  construct the timeline.
  So once we have a list of potential tweet candidates along with their scores, we then
  need to rank them and actually build a timeline.
  So this includes a bunch of business logic to handle things like maybe excluding
  tweets you might have previously seen, determining if there's any social proof to
  display, for example, someone you follow like a tweet or maybe diversifying different
  authors to avoid flooding your timeline with one person's tweets.
  We also have a bunch of like product logic to build conversation modules out of
  replies, ensure there's a balance of different types of content like tweets from
  people you follow versus recommended content, excluding blocked, muted authors,
  things like that. We then inject ads users to follow and handle other timeline
  serving aspects like navigation, instructions or pagination.
  And then we return this final response to the client.
  And now I'm going to hand it back to Tejas.
  Now we have Jawad and Ella, if you want to give some perspective on the trust and
  safety aspect.
  Absolutely. Can you hear me, Tejas?
  OK, so hi, everyone.
  This is Jawad from Trust and Safety.
  So today we are open sourcing part of the code base of two main components of trust
  and safety at Twitter.
  The main component comes from the detection component.
  We are open sourcing the code base, the training code base of a few machine learning
  models, including not safe for work, abuse and toxicity model.
  The second component is called visibility filtering library, which is basically the
  remediation layer of trust and safety with other product services like home terminal
  ranking and forcing Twitter policies, most of Twitter policies, across those
  surfaces. I'm going to pass it to Tejas.
  Hi, Elon, any comments and you want to like have what would be the impact of this open
  sourcing in general for the community and Twitter's future?
  Yeah, sure.
  Well, I see there's a lot more people that have joined.
  So I think it's probably worth me repeating the introduction for those that did not hear
  it. The goal with open sourcing the Twitter algorithm is to build trust and transparency
  with users, build trust through transparency, I should say, in that I don't think you should
  trust any social media algorithm that is a black box and you don't know what's going on
  in there.
  We're trying here with Twitter to really be the most trusted place on the Internet where
  you know you're talking to real people and you know why things are happening on Twitter.
  And it's like the least gameable system on the Internet is our goal.
  Now, our initial release of the so-called algorithm is going to be quite embarrassing and
  people are going to find a lot of mistakes in it.
  But we're going to fix them very quickly.
  And the analog here that we're aspiring to is sort of the great example of Linux as an
  open source operating system.
  Now, even though it's open source and one can, in theory, discover many exploits for
  Linux, in reality, what happens is the community identifies and fixes those exploits and you
  end up having an open source operating system in the form of Linux that is the most robust.
  And in fact, that's what we use as the operating system for Tesla cars and SpaceX, a SpaceX
  Dragon spacecraft.
  So SpaceX uses Linux at the core and so does Tesla.
  And it's because it is so robust and because you know what's going on.
  Because you have the source.
  So that's the intent with Twitter making the algorithm open source is so that the
  community can participate in making it better and identifying issues.
  And then at the end of the day, you can trust what you see and know that it is not
  manipulated or it is the least manipulated information in the world.
  And so I think we can sort of open it up to questions.
  I mean, I think the intent here is for this call, at least, is to focus on technical
  questions. So if you have technical questions, please, I guess, raise your virtual hand.
  Thanks, Ilan.
  I'm scanning the list right now and we're going to start bringing people on stage.
  Great.
  All right. Up first, the whole Mars catalog.
  Hey, yeah, really exciting to see this out on GitHub.
  And I'm excited to see what people find as they pour it over.
  There's been a lot of hype and interest in large language models, and it seems like
  something that is potentially transformative to Twitter as a product recommendation,
  maybe other features.
  Are there anything that the team has thought about?
  Yeah, I think there's a strong argument that the recommended tweets, that what you
  should the optimal thing might be simply to populate a parameter space for any given
  tweet. Then you've got the parameter space for the user and then you just basically
  look for the correlation.
  So you can see, you know, the
  And then you just basically look for the correlation between those two parameter spaces.
  So that that'd be one approach to doing it and probably a significant factor in
  recommendations in the future.
  I mean, the intent is to provide you with recommendations that are as as useful and
  entertaining as possible.
  Our optimization is unregretted user minutes.
  So it's not like how many users do we have or, you know, and or even how much user
  time do we have. But but how many but what's the probable user minutes that are
  unregretted? You know, it's possible to engage in a series of limbic hacks that
  effectively give people a dopamine rush every time they look at some new content.
  But but but if afterwards you regret that and you have somewhat of a hangover from
  it, then that's that's not good.
  We want to maximize unregretted user minutes.
  OK, great. So now let's bring up our next speaker.
  Gonna remove the whole Mars catalog.
  All right, how about we bring up the Doge Designer?
  All right, Doge Designer, you're good to go.
  Doge Designer, are you there? You're free to speak.
  Maybe there's a microphone problem.
  OK, so we'll switch to someone else.
  Slash dot.
  Do I see slash dot on here?
  Like I said, we're kind of looking for technical questions or anything, things
  that where you have a question about the source code and we can try to address it.
  All right, Christina, you're free to speak.
  All right, Christina, fire away.
  Christina, you're muted if you're trying to speak.
  OK, it's like I said, let's try to look for people who have technical questions.
  So if you have a question about the code or something that we can answer, that would
  be helpful as you look at the source code.
  That's that's those the questions which we're aiming to address right now.
  All right, Josh, you're free to speak.
  Hey, you can hear me.
  Yeah.
  Hey, thanks, guys.
  Sorry about the technical issues.
  It's weird.
  Yeah, not the most technical question, but but certainly it does play to the the algorithm.
  Of course, it's the idea of revenue sharing with content creators and just the general
  idea of connecting with content creators.
  And how that's going to be affected by the algo and how you kind of envision that shaping
  with this being sort of crowdsourced now.
  Thanks.
  Well, we're going to look for suggestions, not just on bugs, but also on how the algorithm
  should work.
  Like some of you said, we're going to look for suggestions on how the algorithm should
  work.
  Like I said, the overall goal is to maximize unregretted user minutes.
  So we'll be really interested in any suggestions that are likely to improve unregretted user
  minutes.
  So and this is going to be an evolving process.
  So you and I wouldn't expect it to be a, you know, a nonstop upward improvement.
  I would expect there to be times where it's, you know, two steps forward, one step back.
  But but like I said, we're we're very open to what would improve the user experience,
  what would be a netcode for the world?
  And so there'll obviously be some debate on these subjects and not everyone will agree.
  But at least people will will know what what's going on because we're washing our laundry
  in public, you know, and it's not like if there's dirty laundry, it's not like hitting
  somewhere.
  So that's that's that's kind of but I think that's that's a really big deal.
  So even if you don't agree with something, at least you know why it's like there and
  you're not being secretly manipulated.
  All right, JCS, you're up next.
  You're free to speak.
  I think Ella may have been trying to speak earlier or maybe as well.
  JCS, go ahead.
  Hello. So first, yes.
  Thank you very much for everything that you that you do for free speech, for freedom.
  Yes. Can you hear me?
  Yes. OK.
  First, thank you very much for everything that you do for freedom.
  Second, you mentioned Linux.
  So first, out of curiosity, what is the distro used in Tesla and SpaceX?
  And second, are you how closely are you going to follow that model?
  Are you going to a satellite pull request?
  Because my first pull request will be lightning between invoices.
  We are going to look at pull requests on GitHub.
  So to be clear, like the you know, the questions related to this particular spaces are really
  specifically oriented towards what the code that we're using.
  Oriented towards what the code that we've released.
  So we won't go off subject and go into other areas.
  This is this is purely about the code that's being released.
  If you have questions about it, we were happy to answer them.
  Or if you have recommendations about the code, we're happy to hear those.
  But it will it will operate very similar to Linux with pull requests on GitHub.
  And yeah, so we'll prioritize what we see by what appears to have the biggest impact on the site.
  And I think there's going to be a lot of low hanging fruit at first where there are many issues to fix.
  So it's going to be somewhat of a firefight at first where we figure out what are the most
  important things to fix first and go after those.
  So we're definitely looking for user feedback as quickly as possible so we can take action.
  Thanks. Thank you.
  OK, so let's find our next person to bring up.
  All right, Bitcoin Archive, you're up next.
  All right, Bitcoin Archive, you're free to speak.
  Hi, guys, can you hear me?
  Yes.
  Elon, one of the things that we've been working on for a while now,
  Hi, guys, can you hear me?
  Yes.
  Elon, one of the concerns that I have is that now that you've open source or you're about to open source the code,
  is that everyone's going to start tweeting in much the same sort of formulaic way to optimize engagement.
  How are you going to deal with that?
  Well, we have released the code. It's on GitHub right now.
  There's a link. You can just click on the link at Twitter or on my account.
  So it is there right now.
  I really expect that there will be attempts to exploit what people see in the algorithm.
  So we want to address and patch those as quickly as possible.
  So we've tried to fix everything that we thought was obvious, things that we met.
  So we're looking for feedback from the world and we'll fix it as soon as possible.
  And then, like I said, we'll be oriented towards making changes that maximize cumulative, unregretted user minutes.
  That's our goal.
  So can I have one more question?
  When you mentioned unregretted user minutes, that sort of pricked my ears a little bit,
  because I think Netflix kind of optimized in roundabout the same way, but I'm forever dissatisfied,
  unable to find anything to watch on Netflix.
  And I wonder, is there any more behind that metric that you might be able to share with us that's embedded in the code?
  Well, you can literally see the code for yourself.
  And you'll see what other people who read the code, what their thoughts are.
  So this is, you know, Bitcoin is open sourced as well, which I think is good and contributes to the robustness of Bitcoin mining
  and making sure they're onto exploits in the Bitcoin system.
  So I guess another good example of open sourcing, making something robust, is Bitcoin.
  Thank you.
  Thank you.
  All right, we're going to pull up our next speaker.
  All right, first and foremost, I just want to say shout out to everybody.
  Thanks for adding me up.
  I wanted to ask, first, I wanted to say thanks for everything you've been doing, Eli.
  You know, I'm a big supporter, I appreciate everything.
  I wanted to say basically one question for the code and for the business.
  So like if you're a, I'm an artist, I'm a platinum artist.
  Does that still work for the business code that you're doing, like for the check marks and stuff?
  Like, is that relies to artists or the activity and stuff like that, or is that different?
  Well, the intent of the algorithm is that there should be on an individual basis, it should optimize for the best content.
  So this is, you don't, I think you don't want to have a system where it's like constant pay to pay to play or pay to get pay to get distribution.
  Because then you say you have a fundamental conflict between quality of content and how much money is being paid for content.
  So, you know, I do think it's important that we have only verified accounts in the recommended, but it's, you know, literally seven bucks a month if you buy it through the web.
  Yeah, I bought it, but it'd be glitching.
  Like I bought it, but it's like still in the glitch stage.
  Like it don't say that I got the Twitter boot thing, but it's definitely got me all the other stuff and I've been using it and it definitely has helped like everything.
  So from a user that actually bought it and been using it for since you let it out, it has made my activity on way better.
  I get recommended better. It like unshadowed being me a little bit. That makes sense.
  So I appreciate it. That's all the questions I just wanted to have. I appreciate all you guys for having me up here and I'm a big fan.
  All right. Thank you. I appreciate that.
  All right. Up next, we're going to go with Mary.
  All right, Mary, you're free to speak whenever you're ready.
  Hi. Hello.
  Hello. Yeah, I really didn't expect to get let on. Longtime Twitter user here since March 2009.
  The only thing I miss is the fail well. Make system, make system failures cute again.
  Okay. You can put them in space or something. I don't know.
  Sure. I hopefully we have some failures are hopefully very rare.
  Of course, of course. But like anything's better than making like sometimes I think my account got banned for like a full minute.
  It's like anything's better than that.
  Fair enough.
  Okay. Thank you.
  Great. Thanks, Mary.
  All right. Give me one sec. I'll find our next speaker.
  Actually, while we're finding the next speaker, just to make sure that we're not missing anything.
  Actually, while we're finding the next speaker, just to clearly articulate a point I was making earlier that although we think it makes sense to only have verified accounts recommended in the recommended for you tab,
  apart from accounts that you obviously directly follow, is to ensure that the system is not going to fail.
  Actually, while we're finding the next speaker, just to clearly articulate a point I was making earlier that although we think it's it makes sense to only have verified accounts recommended in the in the recommended for you tab,
  apart from accounts that you obviously directly follow, is to ensure that the system is not gamed by unverified accounts.
  The fundamental challenge here is that it's trivial to create literally 10,000 or 100,000 fake Twitter accounts using just one computer at home.
  And with modern AI, you can bypass every, are you a human check that you can imagine?
  So, so then how do we tell whether an account is real or whether that account people actually like what that account is tweeting or whether it's simply some gamified situation with the botnet and with modern botnets that are really going to be indistinguishable from human.
  So that's the reason for really pressing hard on verified where the verified requires a number from a reputable phone carrier and a credit card.
  This this doesn't make fake accounts impossible, of course, but it I think makes it like literally somewhere between 1000 and 10000 times more expensive to create fake accounts.
  And it also dramatically increases the ease with which we can identify and suspend fake accounts.
  And my prediction is that any so-called social media network that doesn't do this will fail.
  Great. Okay, I think we've got our next speaker ready.
  I believe it's I sciencey you're up next.
  I mean, there's I've seen some some pretty insightful people on Twitter talking about Twitter being like trust as a service.
  And really that that is that is kind of what we're going for is that is to try to be the most trusted place on the Internet in a world where where so many people and entities and governments are trying to influence you and in a lot of cases try to trick you with information that isn't true.
  Well, where where can you go? That is the most trusted or where, you know, where's the information you can actually count on, which is not to say that Twitter is perfect.
  But can it be the least imperfect place to find the truth on the Internet?
  If Twitter can be the least imperfect place to find the truth on the Internet, then I will consider that we've done a good job.
  Great. So I sciencey you're free to speak if you'd like.
  No. Okay.
  All right. All right. Up next, why don't we go with Radar.
  All right, Radar, you're free to speak.
  Radar, you muted yourself.
  Sorry, I just I think one of the things we should
  a side note to change the default on spaces would be that if if we if we tell someone they're about to speak, we should auto unmute their mic.
  I'll talk to the team.
  Hello. Good. Do you hear me?
  Yes. Yes.
  Yes. Yeah, I like the new algorithm that goes open source.
  That's a good idea. And yeah, I find it still weird.
  There's still possible. Yeah, there are many bot accounts on Twitter.
  And I've seen many, yeah, like many, like many people, there's still opportunity to, yeah, to make bot accounts.
  So maybe in the future, it's it's better to detect them.
  Yes, absolutely.
  We've got the great lengths to reduce the amount of bots on the system and to stop scamming and spamming.
  And I think we've made a lot of progress, at least my experience and the statistics that we see for the site.
  We've made dramatic improvements in the sort of spam scam stuff versus, say, five months ago before the acquisition.
  Does this seem accurate to you? Is your experience less bots now?
  Yeah, I'm now using like for like 15 years Twitter now.
  And I've seen I'm living in the Netherlands, but I've seen a decrease for uses in the Netherlands.
  But for America, there's like an increase.
  So, yeah, for me, there's like now less bots now.
  So this makes me more want to use Twitter now because I'm also active on other social platforms.
  For me, I'm a company as well.
  But I prefer more Twitter because it's more easier to to connect with our clients.
  All right. Sounds good. Thank you for your time.
  And everyone, have a nice day.
  All righty, Suhail. You're up next.
  You're free to speak, Suhail.
  So I think you muted yourself.
  Suhail, do you have a question?
  Sorry, can you hear me?
  Yes.
  Yeah. So I guess I'm a developer myself and I just had a question regarding the latest Twitter algo.
  I went to the blog post and GitHub and my question is mainly about the search algorithm.
  Basically, we have the feed algorithm.
  But how does the search algorithm usually work?
  Like, what is the factor that basically the search algorithm work?
  For example, there are different people with the same name.
  And if I search that name, how would Twitter bring up that person that I really want?
  Is it like the path infraction and the number of followers?
  What are the main factors that impact the search algorithm?
  Go ahead, guys.
  Taffy or Jonathan or Brian, what do you want to take that one?
  Well, I can speak a little bit to a similar noted question.
  So one thing is we do use search when it comes to in network tweets, when we want to find tweets from people you follow.
  But it speaks directly to your question of like if you type in a name, finding the actual account that you're interested in.
  So I guess just to note is that to power in network tweets, we do use Twitter search, but it's all from the back end systems where we check for all of the user IDs that you follow.
  And then we hit the search back end, which indexes those tweets for the past 24 hours.
  And then we fetch those for scoring and then determining whether or not to serve that in your for you page.
  But I'm not the expert in terms of how we do user type ahead based on the name to find the account you're actually interested in.
  So maybe how if you have any thoughts on that?
  Yes. Yeah. Hi, this is a healthy. I'm with the LA LA. Can you speak for certain?
  Yes. Well, the search algorithm, a lot of so think about the search as a very also similar with the recommendation system.
  So all the search problem is a very, very complex.
  So we have similar. So we have the similar recommendation kind of steps, just like say we have a retrieval stage.
  We have the ranking stage for the search. And a lot of times a lot of facts in the search actually help you to find the relevance content.
  And so definitely every so the weather wording is matched to each other matters a lot.
  And also, it depends on whether we will have we will have more potential search for the people you are engaged with.
  We try to optimize kind of say what's the most popular things in your areas. All this matters for such algorithms.
  Maybe maybe we can. Does this answer your question?
  Yeah, I believe that. Yeah. Thank you very much. Appreciate it.
  Yes. Just a quick comment. We didn't open source a search in this run.
  This is wrong. It's about more about for you a recommendation.
  We will for sure follow up with open source the entire search as well.
  Great. All right, you stuff. You're up next.
  Oh, do we lose yourself? OK. Onto the next person.
  All right, Natasha, you're up next.
  Hello.
  I was just going to say I think it would be nice if it was easier to report catfish accounts,
  especially for people who are verified to report an account that is stealing or using their photos and post,
  because I feel like currently there is a bunch of hoops to jump through to go through said reports.
  Sure. Great. In general, we need to improve customer service so that especially for verified,
  there is some someone you can communicate with to resolve issues like impersonation or misuse of copyrighted information
  or just any kind of deception or fraud.
  OK, great. Demon, you're up next.
  Yeah, thank you. Thank you for bringing me up and great space, guys.
  And Elon Musk, huge fan and love what you do.
  And just want to say like I had a question about the algorithm in terms of like what we post in terms of tweet.
  Like I know you said like we had like a big emphasis towards media.
  So like, can you just go a little bit on that?
  Well, it's like I said, the goal with what's recommended and we've got a lot of work to do to get there
  is to show you the things that you find most informative, interesting, funny, entertaining.
  And like I said, maximize the number of unregretted user minutes or even really down to the second,
  maximize unregretted user seconds on the site.
  So that's really the goal.
  We do want to minimize for individual content creators.
  Like we don't want to have any kind of like said, pay to play where you have to pay pay for distribution.
  Like your content is really good, provided you are verified that it should be.
  We don't want to be like one of those pay to win games, you know, which really I kind of hate that.
  So it's like, you know, it just makes you feel bad, frankly, to have a pay to win game.
  No, it's just it's kind of like it's kind of like like like Twitter verified,
  which you can get for seven bucks a month if you buy it by the web.
  That's like buying the app.
  But then there's not a whole bunch of pay to play things beyond that.
  I mean, yeah, you can do advertising, but and we have that, of course, and we identify it.
  But for organically recommended content, it should be a function of the quality of the content,
  not how much somebody paid for that to have that content promoted.
  No, one hundred percent.
  Like, you know, also, like I'm like, like I said before, like I'm foolish as fuck.
  Like everything that you do and this verification, like, you know, it was it was great.
  Like the way you rolled out, rolled it out.
  Also, like one suggestion I had, like now that I have them on stage, like for group chats,
  would it be possible to expand like on the seventy five people we have?
  Like it like it really hurts a lot of things like in on Twitter.
  And that would be great if we can just expand on that.
  OK, sure. Like this for DMs and stuff. Yes, yes.
  DMs like right now the cap is seventy five.
  If you could if you guys could like make it like one hundred fifty or whatever, like that would be just amazing.
  OK, sure. Noted. Well, hundred fifty should be quite, quite doable.
  We are rolling out a bunch of improvements in April to DMs, which are admittedly long overdue.
  So you'll be able to reply to an individual DM.
  So like there's a whole bunch of, you know, messages, you know, just we're talking about like basic messages one on one here.
  But we're rolling out the feature next month so you can reply to an individual DM
  and then you can also have any kind of reaction emoji that you want.
  And then it just it just caches the your most used reaction emojis, super basic stuff.
  But then also you'll have the option to turn a chat initially just one on one to turn a chat to encrypted.
  So so you'll lose some convenience if you make a chat encrypted because it's like harder to move that to a new phone.
  So but you'll just have the opportunity to turn a chat to be encrypted or not encrypted very easily.
  Just tap of a button. So that should be really helpful for privacy.
  And then sort of later this year or hopefully not too far in the future,
  probably only a few months, a few months away, we'll be adding in the ability to do a voice DM through DMs to be able to say,
  like, can you speak to someone to have a voice chat or a video chat?
  So can you, you know, if you've got a DM group and you want to turn that into a spaces, you know, private spaces,
  audio chat or a, you know, a video chat, although it gets a little unwieldy at 150 that you can do so.
  But I think it would be kind of cool to be able to use your Twitter handle to have a conversation with someone else without having to give away your phone number.
  So that seems like it could be pretty cool.
  Thanks. All right. I'm next to Sterling.
  Hey, what's up? Thanks for having me out. I've got one technical suggestion and two questions.
  The first, obviously, you guys are, you know, you have this embedding space for how, you know,
  users are related to each other and related to people you follow and stuff.
  And I feel like just giving people a direct user interface to that embedding space would be a huge user value.
  You do that in the Twitter blue, you know, top articles where you can see, like, the top things people you follow are posting,
  but also people they follow, which I feel like is a massive, you know, value add to give people direct access to the way that these graphs are constructed.
  But the two questions that I have, you know, just in the few minutes that I've been looking through this code trying to make sense of it,
  there's a user mass file.
  And it seems like if you're following over 500 people and you're following to follower ratio is above 60 percent, you actually get punished.
  So, you know, if you're following a thousand people and if you have a thousand followers and you're following 800 people,
  you're shown less often than if you unfollowed 300 of those people.
  And like, I get how that could punish people who are following lots of people to spam,
  but it also sort of like punishes curious people who want to use the platform.
  So I feel like that could use some work or looking into.
  And then my second question is there's this file called home tweet type predicates and there's like completely separate types of tweets.
  If author is Republican, author is Democrat, author is power user or author is Elon.
  And I feel like I could see why you might want to categorize those things.
  But just just, you know, political bias aside, dividing people into Democrats and Republicans and like not having more granularity,
  like just seems problematic and it's like reinforcing our two party system,
  which is it's just a bummer to see that sort of stuff explicitly called out and not like an emergent property of some like deep neural networks embedding space.
  Yeah, I agree that that shouldn't be there.
  That's why I preface this by there's going to be a ton of like stupid,
  embarrassing things that are shown here with the, you know, making the code open source.
  The intent is not to say, hey, here is our incredible open source.
  It is a flawless gem. It's like, no, it's it's got a lot of issues that need to get fixed.
  And I think you've identified at least one of them.
  So we want to, you know, basically delete as much as possible.
  That is creating a bias and do it and do it. Yeah.
  Fronto. So, yeah, I mean, it definitely shouldn't be dividing people into Republicans and Democrats or not.
  That makes no sense. The one consistent theme that should be present is are you showing people compelling content
  that, like I said, that they don't regret seeing.
  So, you know, and promoting the public dialogue.
  That's the intent here. So I said, it sounds like you've identified something we should get rid of right away, quite frankly.
  Yeah. Hey, I just wanted to do a quick note on what you brought up, Sterling, which was a great point on the user mass class.
  I just wanted to share some context. So this is under a tweet cred directory, which was an algorithm that was added to the recommendations.
  I want to say about 10 years ago and is actually one of the scores that we're looking to remove where we can, because we don't find it actually provides much value in terms of breaking down.
  So although you've identified how it works correctly, it's actually not too important in how it's used throughout the code base.
  It's primarily as a feature within the in network ranking stage.
  And it's just one of many features and doesn't have that high feature importance.
  But, yeah, we're actively looking to remove that because it's really old and it's not the best way that we could be ranking tweets using that.
  But, yeah, just wanted to call that out. And then how say if you wanted to talk on the other point?
  Yeah, yeah, this is how they and each one will be speaking through my phone.
  Hi, yeah, this is this is the sun. I can provide some context on what you mentioned about the home to be type etiquette style.
  That's purely for metrics collection so that we can track how different groups are doing and make sure that in the experiments we don't penalize certain groups of people more than others.
  Yeah, it's purely just for collecting stuff.
  But we should get rid of it. Yep. Consider it gone.
  Okay, great. Thank you, Sterling. Up next is Youssef.
  Okay, thank you. All right. Salam alaikum during the blessed month of Ramadan. I have a question for the team on Twitter.
  So the question I have is, has there been, I'm sure there has, but I would like to know what the thoughts are on the question regarding removing the whole concept of the algorithm or reducing the concept of the algorithm down to something much more discreet and elegant, which is what Twitter's origin was, which is that essentially the people are the algorithm, meaning that you decide to subscribe to someone you see their tweets in chronological order.
  And then the retweet functionality access the boost or the signal that allows you to see new tweets and interesting tweets.
  So the people are essentially algorithm. And would this also then maybe accomplish the goal of reducing regret?
  Because it essentially turns the algorithm into decentralized individual responsibilities to follow people who are interesting or not.
  So I'd like to know what the thoughts are on that, because that is essentially what Twitter's origin was, right? So what are the thoughts on that simplifying the concept? It still would be an algorithm. It's just much simpler. Thanks.
  Well, I mean, the following algorithm should be exactly that.
  So the intent there is that the following algorithm be, you know, just the people you follow and what they do.
  So it's very easy to switch back and forth between the recommended and the following. You just sort of swipe left and right to go back and forth.
  The reason for the recommended algorithm is just that there are often things that are great that are happening, but you don't know that they are happening or there's something very entertaining, interesting, funny.
  But it's not from anyone you follow. So, you know, how do we expose people to that? And that's what the recommended for you tab is for.
  Now, the following tab is what you described earlier, which is kind of the user generated graph. So, you know, we encourage people to switch back and forth between recommended for you and following.
  Okay, understood. And what is the ultimate incentive, though? Is it about increasing, you know, how much, you know, interesting things you're aware of?
  I mean, how are we knowing that for you is a success? How do we know it's a success? I guess that's what I'm getting at.
  Well, I guess it would depend on are people using the recommended for you column. They're using that a lot.
  And, you know, it should be at this point, I think, sticky. So, in other words, if you prefer to be on the following tab, then it'll leave you there.
  If you prefer to be on the recommended for you tab, it'll leave you there. But like I said, it's just very easy to swipe left and right to switch between recommended tweets and the following user generated follow graph, which is, you know, much more just what people you follow are saying.
  So, but we generally do see a preference, a user preference for the recommended for you over just the user graph, the direct user follow.
  You know, we see a preference for recommended for you versus the following.
  So, we must be doing something reasonably right in the recommended for you stuff, or we wouldn't see a user preference that's quite strongly weighted to the recommended tab.
  Great. Up next is Farzad.
  Yes, thank you. Just want to give everybody a huge props. I think you guys are doing a really good job.
  I've been on Twitter for a while and I've never been more addicted and I don't know if that's a good thing or a bad thing.
  But I really want to give you guys a huge prop, huge props.
  I think the key is, as I was saying, is like, I'm regretted user minutes.
  So, if you're spending a lot of time on Twitter, but it's it but you're, you know, you found the time you, you know, you look back on the time and it was like, you know, you, you learned lots of useful information, and you were entertained, it made you laugh and that kind of thing.
  Which I have to say, I was just thinking about it, like, I get more laughs from Twitter per day than than anything.
  So, so it's, you know, it is working for me. In that sense, it's, it's, yeah, so, as long as the time you're spending is not regretted, then I think that's great.
  Yeah, no, I would say it's been a great place to just find information, you know, it's my key tool for for getting the news and just learning new things.
  So I think from that perspective, it's just, you know, it's, it's, it's hard to step away because there's just so much to explore.
  But I think it's just more about my own self control there. But I think you guys are doing a really good job with that.
  Around the around the trust sort of question, I think one of the big competitive advantages that I see in Twitter around trust is the community notes function, which I think is the really cool tool.
  But like within the context of AI, and you know, this whole mid journey thing that you know, we all know about chat GPT mid journey. You know, there's increasingly more and more sort of things that are coming up that could be very confused for reality.
  And it were very seems like we're so close to just it being indistinguishable from being real life. How are you guys thinking about that solution? Is it scaling up community notes? Is it something else? I'm just really curious to hear where you guys's heads are at with that.
  Yeah, I think we're trying to scale up community notes as much as possible. I think that for important information that is potentially misleading to the public notes is a very powerful means of getting to the truth.
  So we that you know, so ways that we can improve notes are obviously having the notes appear faster. So they you know, it takes you know, if there's some piece of information that's going viral, but it's wrong, then we want notes to fix it before it goes, you know, too viral too fast.
  But but then that's weighed against us having enough contributors to the note and enough validation of the note by trusted contributors to be able to actually make that note visible.
  So now as more people see the note and contribute and comment, that increases the what the sort of credibility score and the sort of meaningfulness score of the of the note and that it will surface faster.
  So but but I do think it's, you know, one of the one of the most powerful things that we've got to get to assess the truth. And the notes apply to everyone applies to me applies to advertisers applies to, you know, presidents of countries.
  Does this notice apply to everything on the site? And, and I think what what what we do see happening, or at least my sort of observation is that once people get noted a few times, they're a little more careful in the future about what they say.
  So, you know, my just, you know, some prominent accounts out there where they've been community noted a couple times, I've noticed that they they now are a little more accurate in what they say, which is encouraging.
  Great. Up next is Abhishek.
  My question is about the Twitter communities. When the feature came out, I was thinking it's, it's a big change. It was supposed to be good for me because I'm into the field of machine learning. So I could just join a community and learn about the new things there.
  I'm into the field of machine learning. So I could just join a community and learn about the new things there. But seems like lately, it's all all spam and bots there in the communities. So are you planning to bring the recommendation algorithms to communities and also recommend communities based on the user's interest?
  There's a whole sort of set of things we've got to solve, because we've got communities, we've got circles, we've got topics. And, you know, there's so many different things and lists. So now, probably the best way to get information about a specific topic is to get information from the community.
  The best way to get information about a specific topic is actually to create a list. So if you get a Twitter pinned list, the pinned list will then appear as a menu item at the top. So you can, you can swipe right from for you to following to whatever list you create.
  That I think is the best way to follow detailed information on a particular subject without being subject to bots spam.
  But we need something we need to do is we need to make it easier to create a list and to recommend high quality lists in particular subject areas.
  Thank you. Okay. Thank you Abhishek. Up next is Mahdi.
  Hey, there is a question about a piece of code in the report that you've just published about a line of code that is labeling some tweets that the author is along. Can you please explain about that?
  I mean, yeah, guys, what's going on?
  And yes, we do release the code that it's author is along and author in different groups. If you read the code carefully, that's only for stats tracking purposes. And it has nothing to do with the algorithm. It's only for stats tracking purposes.
  The reason why we put it in to make sure that the product changes that we have made, no matter it's under the algorithm or user experience, and we want to make sure we don't bias towards one group versus another one.
  So that's why the group exists. So yeah, that's why the code is there. So there's no special treatment on algorithm at all.
  Yeah, I've already seen that part of the code that is talking about the metrics, but isn't it weird that you have four categories and one of them is along?
  I think it's weird. Yeah.
  I mean, this is the first time I'm learning this, by the way.
  So that's the privilege of being open source. So you can see it on GitHub now.
  Yeah, that's one of the big reasons that we're looking for feedback. We're looking for areas to improve and we'll make improvement aggressively.
  We'll open source. Thank you. Thank you.
  All right, Elon. At a glance, I think we've spoken to a lot of the engineers that have their hand up on the call.
  We can continue going or opening up to other comments, but just want to pulse check.
  I think this is good. Thanks everyone for your feedback.
  And yeah, we'll be making a lot of changes to the Twitter recommendation algorithm based on the recommendations that you make about changing it.
  So expect a lot of bug fixes and modifications at a really high clip in the weeks to come.
  Thank you for your feedback and helping to make Twitter a more awesome place to be. Thanks.
  Good. Thank you, Elon. Thank you everyone for joining.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.
  Thank you.