Bevy 0.10: How do I render 3D text in bevy? (using font.ttf file)

Question

I am developing a card game for which I need to render some 3d text. The game itself is designed to be 3d, however the card graphics are .png files (so effectively 2d) which I am rendering using the shape::Quad + StandardMaterial.base_color_texture to asset_server.load("cards/example.png") on a PbrBundle. I don't know any other way of rendering effectively 2d content (e.g. icons) in a 3d context, this technique was even used in the official bevy examples to render text here.

My problem comes when I want to render text, e.g. numbers, as part of the game. Ideally, I would like to spawn an entity with a Transform and a custom component which I can Query<&mut CardNumberComponent> to update, which is rendered by my Camera3dBundle as a 'normal mesh' alongside other entities (mostly PbrBundles), but is rendered as text. For example, I want to create a 'card' entities which aesthetically totals to a 2d image (normal PbrBundle + custom material as described above) and a health total, the health total being specifically a number that is part of the 3d 'card' entities (probably a child), so when I move the card the number moves with the card (like a child PbrBundle entity moves with its parent entity as shown in this official bevy example).

My current attempts to achieve such "Text3dBundle" like behaviour have partially succeeded, if I know the range of possible numbers (or more generally text) that I need I can pre-generate images and call asset_server.load(format!("cards/nums/number_{}.png), x)). This works, but I wonder if I have simply missed some simple API provided by bevy or a third party crate.

Stated formally:

My problem requires this function, using bevy 0.10.1 (from crates.io) and compatible with macOS + WASM (trunk serve should 'just work'), spawns text into the bevy world:

pub fn render_text(
    at: Transform,
    content: &str,
    font_file: &str,
    commands: &mut Commands,
    asset_server: ResMut<AssetServer>,
    materials: ResMut<Assets<Mesh>>,
    meshs: ResMut<Assets<Mesh>>,
// other bevy system parameters
) {
    // load font
    let font = asset_server.load(font_file);

    // create text
    let text_something_idk = todo!();

    // spawn in
    commands.spawn(text_something_idk);
}

That is my problem, how do I render 3d text in bevy?

Almost solution, WASM not supported

One way of solving this is to render the text you want + font file into a .png file at runtime as required. Then, theoretically you could save the data as a file locally and then load the same file as an asset in bevy. One such (working) implementation is this: (cargo add text-to-png@0.1.1)

fn text_to_image_runtime(text: &str, asset_server: Res<AssetServer>) -> Handle<Image> {
    use text_to_png::TextRenderer;

    let renderer = TextRenderer::try_new_with_ttf_font_data(include_bytes!("font.ttf"))
        .expect("Could not load file");

    let text_png = renderer
        .render_text_to_png_data(text, 42, "white")
        .expect("Couldn't custom render text");

    // NOTE: This line will break on WASM
    std::fs::write(format!("assets/text-file-{}.png", text), text_png.data).expect("File works + not on WASM");

    asset_server.load(format!("text-file-{}.png", text))
}

pub fn render_text(...) {
// using PbrMesh with StandardMaterial.base_color_texture = text_to_image_runtime(text)
}

The issue is, I need my game to support WASM targets and std::fs is basically unavailable in the browser, so I have no way of passing the png data to the bevy asset_server without using asset_server.load so that I can render the custom png as a texture as a text representation in my game. If only there was an let img: Handle<image> = asset_server.load_from_bytes(input_bytes) API, this solution could work.

Answer 1

Using the mesh text crate, this work:

fn spawn_text(
    text: &str,
    center_pos: Transform,
    colour: Color,
    pixel_size: f32,

    commands: &mut Commands,
    (meshs, materials, _): &mut ASS,
) {
    let (mesh, offset) = get_text_mesh(text, pixel_size);

    // text
    commands
        // parent is a ChildBuilder, which has a similar API to Commands
        .spawn(PbrBundle {
            mesh: meshs.add(mesh),
            material: materials.add(colour.into()),
            // transform mesh so that it is in the center
            transform: center_pos.translate(offset),
            ..Default::default()
        });
}

/// Returns mesh + offset (to ensure coordinates start in center of text)
fn get_text_mesh(text: &str, pixel_size: f32) -> (Mesh, Vec3) {
    use meshtext::{MeshGenerator, MeshText, TextSection};
    let font_data = include_bytes!(concat!(
        env!("CARGO_MANIFEST_DIR"),
        "/assets/fonts/Oswald-Regular.ttf"
    ));
    let mut generator = MeshGenerator::new(font_data);
    let transform = Mat4::from_scale(Vec3::new(pixel_size, pixel_size, 0.)).to_cols_array();
    let text_mesh: MeshText = generator
        .generate_section(text, true, Some(&transform))
        .unwrap();

    let vertices = text_mesh.vertices;
    let positions: Vec<[f32; 3]> = vertices.chunks(3).map(|c| [c[0], c[1], c[2]]).collect();
    let uvs = vec![[0f32, 0f32]; positions.len()];

    let mut mesh = Mesh::new(bevy::render::render_resource::PrimitiveTopology::TriangleList);
    mesh.insert_attribute(Mesh::ATTRIBUTE_POSITION, positions);
    mesh.insert_attribute(Mesh::ATTRIBUTE_UV_0, uvs);
    mesh.compute_flat_normals();

    (mesh, Vec3::X * (text_mesh.bbox.size().x / -2.) + Vec3::Y * (text_mesh.bbox.size().y / -2.))
}

Answer 2

Since you explicitly mentioned

but I wonder if I have simply missed some simple API provided by bevy or a third party crate.

I will provide a sample on how this could be done using my crate meshtext:

use bevy::prelude::*;
use meshtext::{MeshGenerator, MeshText, TextSection};

fn main() {
    App::new()
        .insert_resource(Msaa::Sample4)
        .insert_resource(ClearColor(Color::Rgba {
            red: 1f32,
            green: 1f32,
            blue: 1f32,
            alpha: 1f32,
        }))
        .add_plugins(DefaultPlugins)
        .add_startup_system(setup)
        .add_system(rotate_system)
        .run();
}

#[derive(Component)]
struct RotationEntity;

/// set up a simple 3D scene with text
fn setup(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<StandardMaterial>>,
) {
    let font_data = include_bytes!("../assets/font/FiraMono-Regular.ttf");
    let mut generator = MeshGenerator::new(font_data);
    let transform = Mat4::from_scale(Vec3::new(1f32, 1f32, 0.2f32)).to_cols_array();
    let text_mesh: MeshText = generator
        .generate_section(&"Hello World!".to_string(), false, Some(&transform))
        .unwrap();

    let vertices = text_mesh.vertices;
    let positions: Vec<[f32; 3]> = vertices.chunks(3).map(|c| [c[0], c[1], c[2]]).collect();
    let uvs = vec![[0f32, 0f32]; positions.len()];

    let mut mesh = Mesh::new(bevy::render::render_resource::PrimitiveTopology::TriangleList);
    mesh.insert_attribute(Mesh::ATTRIBUTE_POSITION, positions);
    mesh.insert_attribute(Mesh::ATTRIBUTE_UV_0, uvs);
    mesh.compute_flat_normals();

    // text
    commands
        // use this bundle to change the rotation pivot to the center
        .spawn(PbrBundle {
            ..Default::default()
        })
        .with_children(|parent| {
            // parent is a ChildBuilder, which has a similar API to Commands
            parent.spawn(PbrBundle {
                mesh: meshes.add(mesh),
                material: materials.add(Color::rgb(1f32, 0f32, 0f32).into()),
                // transform mesh so that it is in the center
                transform: Transform::from_translation(Vec3::new(
                    text_mesh.bbox.size().x / -2f32,
                    0f32,
                    0f32,
                )),
                ..Default::default()
            });
        })
        .insert(RotationEntity);

    // light
    commands.spawn(PointLightBundle {
        point_light: PointLight {
            intensity: 1500.0,
            shadows_enabled: true,
            ..Default::default()
        },
        transform: Transform::from_xyz(4.0, 8.0, 4.0),
        ..Default::default()
    });
    // camera
    commands.spawn(Camera3dBundle {
        transform: Transform::from_xyz(0.0, 2.0, 6.0).looking_at(Vec3::ZERO, Vec3::Y),
        ..Default::default()
    });
}

fn rotate_system(time: Res<Time>, mut query: Query<(&mut Transform, With<RotationEntity>)>) {
    for (mut transform, _) in query.iter_mut() {
        transform.rotate_y(time.delta_seconds() as f32);
    }
}

Which will give you this:

The full sample can be found here, including the sample application running on wasm.