This document is a collection of ideas related to improving Bevy UI at the foundational level. These ideas are chosen to be ones that will not interfere with large-scale efforts to develop a UI asset and templating system. This means that the items listed here either either template-agnostic, or are general enough that they would conceivably mesh with any future templating system being contemplated.
This section establishes a set of working assumptions. We don't know yet what form style assets will take, but we can assume the following:
- Style assets will be a thing.
- The design of style assets won't be a clone of CSS stylesheets, but will take some ideas from CSS.
- Style assets will consist primarily of collections of style properties.
- Multiple styles can be applied to a UI element, such that the actual style represents the sum or composition of the individual styles.
- There will be some mechanism for dynamic styling, in other words, styles that are conditional on the state of a UI element.
This is a fairly loose set of assumptions, but even this much lays some constraints on how UI node elements should interact with styles.
Currently, Bevy styles are split across multiple components: The BackgroundColor component stores the fill color of the widget's rectangle, while other components store attributes such as margin width or border thickness. Having style properties be split across components makes sense for manipulating widgets at an API level - there's no point in having a BackgroundColor field if you aren't drawing a background.
However, it's cumbersome if each of those different types of components need to be a separate asset. This gets worse if an element has multiple style overrides, which will often be the case, resulting in a combinatorial explosion of style objects.
It may be more artist-friendly to hide this complexity and provide a more CSS-like experience, where all possible style properties are available in a single context, one which is later split up by Rust logic. A StyleAsset asset might be a unified bag of properties which can contain both background-color and margin-left.
(There's a counter-argument to this: hiding the complexity of how styles are represented only makes sense because styles themselves are complicated, due to the compositional / override / dynamic aspect of these assets, which by long tradition specify behavior declaratively in data rather than in code. For other kinds of entities, with less abstraction between asset and ECS graph, it's possible that we want the artist to see exactly what the resulting entity graph looks like.)
Not every style property applies to every type of UI element. The font property only applies to text spans; text spans, on the other hand, can't have a flex-grow property.
There's also an interaction between components and dynamic styles - an element might have a BackgroundColor or Border part of the time, but not always. A style might specify a background color, and a later (override) style might remove that, changing the background back to the default transparent. Because we don't want to be adding and removing components unnecessarily, we'll want to compute the "final" background (that is, the composition of all background properties) before we actually start making changes to the entity and its components.
In the browser world, there's something called a "computed" style, which is the composition of all styles that are applied to an element. We can envision something similar in Bevy:
flowchart LR
subgraph Temporary
ComputedStyle
end
subgraph Assets
A[Style 1] --> ComputedStyle
B[Style 2] --> ComputedStyle
C[Style 3] --> ComputedStyle
end
subgraph Entity
ComputedStyle --> BackgroundColor
ComputedStyle --> BorderColor
ComputedStyle --> Style
end
In this design, a temporary object "ComputedStyle" is used to gather all of the style properties from the various styles. Once we have iterated through all of the styles, we can then update the properties of the entity's components, including adding and removing components as needed.
In this design, the ComputedStyle is a large struct containing all possible style properties. This isn't the only choice, however, as one could imagine a different architecture where there are multiple flavors of ComputedStyle, one for backgrounds, one for borders and so on, each one having only the properties relevant to that aspect of styling. This reduces the size of the structure, but requires multiple passes through the list of input style assets.
The following features are intended to make it easier for someone to implement something like ComputedStyle.
Currently, there are several UI-related bundles: NodeBundle, ImageBundle, AtlasImageBundle and so on. However, a dynamic styling system probably isn't going to any of use these. Instead, the choice of whether to include a "background" or "image" component is going to be determined by which style asset properties are present. If there's a border-color property, then the BorderColor component will be present, otherwise it won't be.
However, some components, such as GlobalTransform and InheritedVisibility are required for any UI element. So there should be a bundle that contains only the required components and nothing else - so that the styling system can decide what other components should be present.
(Ref issue bevyengine/bevy#9725 and bevyengine/bevy#7616). Currently in Bevy, each text span must specify a font handle. The problem with this approach is that bold and italic font styles require either choosing a different font asset, or choosing different rendering parameters, depending on how the font is constructed.
It would be better if text styles could be specified independently from font faces like they are in CSS: "I want this paragraph to be 'Arial, 10pt', and I want this word to be bold.". This requires a registry of fonts, and a means for mapping text styles into font handles.
(Since there are already open tickets on this issue, I'm mainly mentioning it here for completeness.)
We should not expect that style assets will use built-in serialization for Bevy style properties such as margin-right or flex-grow. The reason for this is that style assets will likely have additional features that also need to be serialized along with the values. Take for example, the property display, which can be one of Display::grid, Display::flex and so on. A style asset system will want to be able to read and write these properties; but they may also want to assign other values to the display field, such as:
initial- resets the property back to its base valueinherit- sets the property based on the setting of the widget's parent (only valid for some properties like text color)- a reference to a variable.
- an expression or calculation.
However, even if the style asset implements its own serializer which supports these additional features, it may still be valuable to have some help with encoding and decoding the various options. AlignContent, for example, has 10 different values (start, end, space-between and so on), and simply being able to convert to and from a &str representation would reduce a lot of boilerplate.
Game UIs tend to be both art- and animation-intensive. Web developers have long had access to a rich set of animated transitions using simple syntax. Eventually we'll want declarative transitions triggered by dynamic style updates, but as a prerequisite we can think about something more basic: a framework for manually triggering animated style properties.
The animation system that Bevy currently provides for things like character animation is considerably more complex than the one used by the vast majority of CSS transitions on the web. The goal here would be to provide a simpler API - either based on the existing Bevy animation system, or a new one - that can provide simple transitions between two states with cubic or quadratic easing curves.
The most "interesting" UI properties from an animation standpoint are:
- 2D translation
- 2D rotation
- 2D scale
- 3D perspective transform (rotations in 3-space for example).
- opacity
- color
While animation of other properties may be useful, the above properties are the critical ones.
It's common in UI designs for overlay items such as dialogs, pop-up menus, toolips or other overlays (even major game modes) to "fade in", that is, to have an enter / exit transition that involves animating the opacity of the overlay. Unfortunately, this is hard to do in Bevy currently, for a couple of reasons:
- While it is possible to modify the alpha channel of the background color, border color, and text color of individual elements, these parameters all have to be modified separately. There's no overall "opacity" setting for a UI graph.
- Even if you did change the alpha channel for each ui component, it would look bad, because now the background parts of the overlay which are meant to be hidden can show through the foreground elements. For example, imagine a dialog with a button that has white text on a red background. As this dialog fades out, you want the alpha of the white text to diminish, so that you see the translucent white against the screen backdrop; what you don't want to see is pink text which is the result of 50% opaque white drawn on top of 50% opaque red.
What you really want in this case is to render the entire dialog onto a separate render target, and then composite in that render target to the main viewport. The alpha channel of the composite operation can then be animated so that all of the overlay contents get faded in uniformly.
Other rendering parameters for the compositing step can be animated as well, for example animating blur via a custom shader. And while animating transforms on the individual UI elements (scale, rotation and translation) is fairly easy, it might be desirable in some cases to animate the transform of the composited image instead.
Ideally, setting up this kind of "buffered rendering" for UIs should be as simple as adding a special Component, but in practice it's likely to be more complicated than that, requiring setting up a second UI camera and allocating a render target.
So the task here is to add features to Bevy UI to make the processs of rendering UI to an offscreen target as simple as possible.
A common design pattern in games is "nine-patch" images, where some portions of the image stretch along one or both axes and other portions remain constant in size. While there are crates that add nine-patch capability to Bevy, these aren't really necessary except as a convenience. It's possible to implement a nine-patch button using only grid layout and a texture atlas - the button simply has nine children with their flex attributes set appropriately. The only problem with this is that it's a pain to set up, because you have to create the nine children, set up the texture atlas regions, and so on. It gets even worse if your button has different states (pressed, hovered, focused, disabled) with different artwork for each state.
Once we have UI templating, a lot of this complexity goes away - the button template can include the nine children and all of their style properties and texture references as needed. However, if UI templating is a long way off, then it might be valuable to build a UI example showing how to set up a nine-patch button manually.