Hi folks,

Usual Disclaimers

These posts will often contain talk about future work we're doing, or planning to do, that isn't yet guaranteed to ship. The nature of the work could change or, depending on what we discover, projects mentioned may get delayed or even stopped. If you'd like to see a Tweet whenever a new one of these posts goes up: https://twitter.com/RiotMeddler

How do we assess our own work part 1: Gameplay

Today I wanted to start covering something a bit different, sharing some details with you all on how we judge our own work. My hope is that makes it a bit easier to understand our approach on some things, gives more shared context for future conversations and is potentially just of general interest to a few people too. This time we’ll cover MOBA gameplay focused teams. Future posts will then look at TFT, personalization/cosmetics, tech, some aspects of marketing and player communication etc.

These posts will be broken down roughly by team though not in a way that's fully representative of our internal structure since I've focused more on teams that have very player facing impacts (e.g. game balance) and less on those that are more internal facing (e.g. tools development). It also doesn't cover some aspects of League of Legends that are handled by other parts of the organization (e.g. Esports is a separate part of the company to the LoL team, chat moderation is done by a cross game group as is player support etc). It also doesn't cover all the things we track, just the key points.

With that out of the way first up we've got Champion Team. Here we're looking at how players engage with a new champion or VGU, how that release impacts the wider LoL ecosystem and how maintainable the new/updated champ is. To assess those things we look at measures like:

• Play rates for a champion, especially how many average games a player puts into them if they play them in the first place (depth of appeal with mains). How many people play them (breadth of appeal) is also of interest, though not to the same extent as depth. Focus is on long term interest, not just interest around launch time.
  • We're not aiming for every champion to have high breadth and depth of appeal, having some niche champs who are really loved by their players is fine. We don't want champs are neither broad nor deep though (champs few people play and those who try them don't play them much)
• Player surveys on the champion's theme, personality, art quality etc
• Tracking rate of balance changes and bug fixes needed post champion release
• Internal assessments on complexity, technical risks, efficiency of development etc
• Degree to which the champion broadens our IP, letting us tell different types of stories, fleshing out factions, better representing under represented groups (e.g. our roster still skews towards male champs even though our more recent releases have had a better gender mix).

With the Summoner's Rift Team (sometimes referred to as the balance team, though they do a lot more than just balance) we're focused heavily on providing a compelling, fair and deep gameplay experience. That means game balance, choice structures, counterplay, creation of highs and lows, variety of play experience etc. Key things we look at to assess that are:

• Champion Balance for existing champions (not new releases, champion team are responsible for a champion's numbers on launch).
• Game pacing, looking at things like how snowbally games are, when towers fall, amount of PVP action etc
• Player surveys on things like champion frustrations, rate of gameplay change, champion damage output and tankiness etc
• Correlations between gameplay changes and levels of player engagement including looking at player desire to queue up for the 5 different positions in LoL
• Balance and meaningful choices for supporting systems like Items and Runes

Our next gameplay team is the Game Analysis Team, also known as the playtest team. Their role is to be a source of expertise on LoL, able to provide strong analysis, repeated playtesting on content followed by actionable feedback and demonstration of what high skill play on upcoming content, features or adjustments probably looks like. To assess that work we look at:

• Ability to predict impact of balance changes based off internal playtesting and theorycrafting (SR team is responsible for making balance changes, GAT for providing predictions on their impact which SR team then determines whether/how to act on)
• How predictions fare for new champ/VGU launch balance and change in balance over time as players become familiar with the kit
• Ability to quickly identify problems during development of content and systems letting teams working on those features avoid wasted work going down the wrong path (e.g. a new game mode's most optimal way of being played is extremely unenjoyable, leading to a no win situation for players)
• Overall value provided to the teams whose work they analyze

Up next is the Competitive and Meta Systems Team. This is a group that are focused on some of the around game experiences like competitive rankings, some aspects of progression systems, some areas where player behavior and gameplay intersect etc. To judge success here we look at things like:

• Ranked engagement like how many players play this season versus last, how does their engagement change over the course of the season etc
• Clash engagement, including how many players try Clash and how many return for subsequent tournaments
• Matchmaking quality which includes how small the MMR differences between players matched together is, time taken to match players, proportion of players who get their first choice of role, autofill rates etc
• Player Behavior metrics around things like AFK rates, frustration with negative behaviors from other players (note: this has only recently been taken on by this team, another group was previously responsible for it)
• Player sentiment on features like Ranked, Clash, Matchmaking etc

Finally we've got the alternate game modes team who are responsible for MOBA experiences that aren't standard Summoner's Rift. That means ARAM and event modes like URF, One for All and Nexus Blitz. When looking at work in these spaces we consider:

• Play rates of those other modes, both average engagement and spikes in engagement
• Player sentiment on alternative modes, including for event modes whether said modes are a key part of what excited players about an event (whether or not they played a lot of it was it something they were excited for?)
• How much alternative modes drive new engagement (players coming back after an absence or playing more because of the mode) versus how much they just move hours from one mode to another

Quest Skins

In the coming weeks, you may start to see the phrase “Quest Skin” floating around, so we wanted to provide some context around what that means. We are testing a new skin type (not a new skin tier with a different price) that will be purchasable in the near future. It’ll have some new features with specific technical needs which is why we need to have it on PBE and tested in advance. It won’t be launching this patch or in the immediate future, but stay tuned for more information!