lip syncing dialogue any tips?

Question

OK here is the situation - I have an animated video which has some close ups on the face, the lip sync is out. In some cases the dialogue if it starts in the right place will be out of sync by the end of the line. One can stretch this in using pro-tools elastic time, - no problem - but getting it to actually feel right is another matter. So my question to you guys is this:

Does anyone have any tips on how to make the lips sync feel right?

For instance do you have any particular speech sounds that you try to sync first to get the right effect, or do you feel that starting the dialogue early by a couple of frames makes the lip sync feel better? Any tips or tricks would be greatly appreciated.

Update:

I think we have found the problem and come up with a solution and I thought I would share it with you guys as you have all been so helpful. I spent a day cutting and moving each word to get them in sync with the animation usually by about half a frame one way or another (using a mirror matching the P's abd B's and not time stretching at all ;)). I then took my results and findings to the animators and showed them the issues. To cut a long story short, the problem was to do with frame rate, the simple explanation was the animators were working at 50 frames, but the game engine plays back at 30. The animation curves for the face were alaising from the 50 to the 30 meaning they were 'missing/averaging' 2/5ths of the information. So the solution to this problem is they are going to re animate at 30 fps and that should solve the majority or the issues. This is a great relief for me as I do not envy anyone who has to try and match lip sync by hand. It is also great that it is not always the case of 'fix it in post' -

Answer 1

Work with a mirror beside your monitor and that way you can see what the mouth shapes should be for every word.

Answer 2

Is there guide audio? The animators "should" have had scratch dialogue in order for them to draw the lip movement.

I'm not totally sure of your workflow, but sync has to be achieved when you're recording your actors. You can cut and paste and squeeze and stretch dialogue all you want, but if you process it too much, it'll just sound wrong. If there is a scratch track, that can help you get good sync when you're recording your voice talent.

I do a lot of animation dubbing, and the only way to get it right is to record it right (or within about 5%, which can be taken care of with some light time compression/expansion).

Answer 3

You might have a speed problem. Is this PAL or NTSC? If it's too fast try resampling at 46034 Hz. If too slow, resample at 50000 Hz. Sync with plosives like P or B, also M's will work. You should not have to re-sync each word.

Answer 4

Frame by frame, line by line.....You have to accept the tedium of the task. That's the price for an inaccurate performance (if the sync was done after the picture).

Elastic Audio as well as some other TCE tools such as VocAlign will work...but I would cut the "lines" into the smallest possible pieces - using TCE as minimally as possible.

Answer 5

B's and P's baby. To a lesser extent F's, M's and V's.

Answer 6

I have never had to attempt this, as most of the time the sync has been checked at the ADR stage or has been done properly by the animator.

From working with students who have recorded ADR that is a little out, I have asked them to look for Hard letters (technical term?); these give distinctive pops and good visual mouth references.

Rather than going for elastic audio, which still seems to colour the sound of dialogue IMO, I would prefer to edit bad dialogue over a couple of tracks and route to a group aux for better sync and overlap.

Answer 7

In 15 years of editing and mixing animation, I've found that animators SELDOM get the lip sync exactly right, for a wealth of reasons, some of which the animators have little control over including time budget, and poor character designs. As far getting things to "feel right", it really boils down to just nudging things. I've moved lines of dialog 4 or 5 frames to make it feel right, even when things were "in sync" with the start of the audio. There are two main reasons.

One, animation uses only a limited number of "mouth positions"... anywhere from 2 (open and closed for very crude animation) to as many as twelve. Since the human mouth makes many more than that, often animators have to match as close as they can. Sounds like "s" can be difficult, since the mouth moves very litte. If you really pay attention to the way humans speak, you'll notice we can make a HUGE quantity of sound, while moving our lips very little.

The other reason is that animation is very often done "on twos". In other words, motion only happens every TWO frames.. hard consonants like a 't' can sometimes last only a frame, which makes things seem out.

Unfortunately, there's no hard and fast rule as to when to move forward, and when to move backward. A lot of my time doing Dialog sync in animation is spent moving things two frames this way, and then three frames that way until I find it "works". Having said that... most people notice dialog that's too early, before dialog that's too late (because we're used to sound travelling slower than light), so I usually start by nudging the dialog later... If that doesn't work, then I nudge it sooner.

Answer 8

I do not agree! If the acting isn't close enough, it will sound and feel awful. Feel beats sync! When it's close already THEN you can align it. But if it's way out I'd go old school and match the end of the line.

Answer 9

Hi!

As someone said above, P's and B's are great points of sync. If the animation is close enough, you should place the very beginning of the "P" sound in the first frame the characters' lips are preparing for this letter. Vowels, most of all "A" and "O" are probably the best for stretching. I advise you, if you haven't went through yet, Jay Rose's Audio Post-Production for Film and Video.

Good luck.

Answer 10

imho

There are only two rules:

1) Just begin to do it and most of your questions will disappear.

2) On editing (sync) stage your primary task is to sync the dialogue. Lips by lips, frame by frame and that's it. Later on re-recording stage re-recording mixer will do it so that it will feel right.