CODEWITHSUNDEEP
unity-game-engineanimation2dtext-to-speech
Luna
Luna

Reputation: 11

Text to 2d Lipsync In Unity

I'm very very new with unity, and I'm not really sure where to start. I'm trying to make a text to 2d lipsync using unity. I've tried searching for tutorials, but all I found was people using a ready made script or plugin to make something similiar, but I want to try to make it from scratch. It's just I only know the basics in coding, and can only modified an existing code and doesn't really have the ability to make anything from scratch yet because I'm still learning, so I'm wondering if you guys can give me a clue?

For example, if you guys can point me to some kind of tutorial for making text to speech from scratch, or tutorial for making 2d lipsync, or some kind of sample code for making text to speech, or sample code for making 2d lipsync, whatever it is, as long as it is related to making text to 2d lipsync in unity, I will gladly accept and study it. Thank you in advance. (Apologize if there are many mistakes in my English, I'm not a native person)

This is the example of what I'm trying to make:

(the problem with those two is that they make the lipsync detect the speech from audio instead of text)

Or something like this but with 2d instead :

I'm sorry if my questions sounds dumb. I'm just very desperate:') any sort of help would be very much appreciated.

Upvotes: 1

Views: 1156

Answers (2)

Codemaker2015
Codemaker2015

Reputation: 1

You can try SALSA LipSync Suite V2. SALSA (Simple Automated LipSync Approximation) is a real-time system for creating lip synchronization from audio input without the need for phoneme mapping or baking. This creates a simple and fast workflow that allows the designer to implement lipsync into their project without spending a lot of time or effort. You can do lipsync for both 2D and 3D characters using this package.

  • Import SALSA LipSync Suite V2 from the asset store
  • Open the SALSA-basic3D-boxhead.scene scene.

https://assetstore.unity.com/packages/tools/animation/salsa-lipsync-suite-148442 https://crazyminnowstudio.com/docs/salsa-lip-sync/modules/overview/

Demo app repository link: https://github.com/codemaker2015/SALSA-Basic-3D-Demo

You can try the Avatar SDK to do lipsync on a real avatar.

https://docs.avatarsdk.com/unity-plugin/2.2.1/using_salsa_lipsinc_v2_with_avatar_sdk.html

Upvotes: 0

tenn1440
tenn1440

Reputation: 41

Depending on your use case you can probably do this a lot more simply than true text to lip-sync functionality but first -

When you're just learning to code a lot of things can be overwhelming at first. No programmer solves the large problem all at once. You want to first break it out into chunks, break those chunks down into further chunks, and tackle them one at a time like a checklist.

For this project, I would break things out like this.

  • Visuals
    • 2D sprite images for the animation (the example you sent only used a few different images and just cycled through them)
    • Sprite animations
    • Sprite animator & controllers
  • Text logic controller
    • text ui updating
    • text controller to trigger various animations

Another important part of programming is working in phases. Determining what your minimum viable product is and iterating on it after you have your basic proof of concept working.

Right now, for your MVP all I would do is create a basic talking animation and then just set up your text controller so that any time text is on the screen, it animates it for X period of time.

Lastly, there is no shame in using plugins or other people's code. Even the best programmers use frameworks or other peoples code when it makes sense to. There's no reason in reinventing the wheel as long as the code is open source or licensed for you to use as needed. Plugins and other's code can also be a great way to learn how to code by seeing how they did it, deconstructing it, and making it your own.

Edit: If you're just looking for a way to enable your animation when the text shows up, add something like this to your text controller or whatever you're using to display the text.

public Animator myAnimator; (drag and drop your animator onto this in the inspector)

public void UpdateOnScreenTextDialogue() {
    // logic to update your GUI with the new text
    myAnimator.Play("nameOfTalkingAnimation");
}

You'll also need to handle resetting the animation or going to specific ones but that can all be looked up in the unity docs or if you google starting/stoping animations in Unity.

Upvotes: 1

Related Questions