4

OK here is the situation - I have an animated video which has some close ups on the face, the lip sync is out. In some cases the dialogue if it starts in the right place will be out of sync by the end of the line. One can stretch this in using pro-tools elastic time, - no problem - but getting it to actually feel right is another matter. So my question to you guys is this:

Does anyone have any tips on how to make the lips sync feel right?

For instance do you have any particular speech sounds that you try to sync first to get the right effect, or do you feel that starting the dialogue early by a couple of frames makes the lip sync feel better? Any tips or tricks would be greatly appreciated.

Update:

I think we have found the problem and come up with a solution and I thought I would share it with you guys as you have all been so helpful. I spent a day cutting and moving each word to get them in sync with the animation usually by about half a frame one way or another (using a mirror matching the P's abd B's and not time stretching at all ;)). I then took my results and findings to the animators and showed them the issues. To cut a long story short, the problem was to do with frame rate, the simple explanation was the animators were working at 50 frames, but the game engine plays back at 30. The animation curves for the face were alaising from the 50 to the 30 meaning they were 'missing/averaging' 2/5ths of the information. So the solution to this problem is they are going to re animate at 30 fps and that should solve the majority or the issues. This is a great relief for me as I do not envy anyone who has to try and match lip sync by hand. It is also great that it is not always the case of 'fix it in post' -

10 Answers 10

4

Work with a mirror beside your monitor and that way you can see what the mouth shapes should be for every word.

  • That's a great idea! Never thought of that! – Audiophile.2010 Feb 14 '12 at 17:59
  • @Iain McGregor this is a good idea thanks :) – RedSonic01 Feb 14 '12 at 18:05
  • I've used my iPhone to record myself saying a line with the video camera to find sync on a line a couple times. Worked out well. – Bill Mellow Feb 15 '12 at 0:41
3

Is there guide audio? The animators "should" have had scratch dialogue in order for them to draw the lip movement.

I'm not totally sure of your workflow, but sync has to be achieved when you're recording your actors. You can cut and paste and squeeze and stretch dialogue all you want, but if you process it too much, it'll just sound wrong. If there is a scratch track, that can help you get good sync when you're recording your voice talent.

I do a lot of animation dubbing, and the only way to get it right is to record it right (or within about 5%, which can be taken care of with some light time compression/expansion).

  • @Roger Middenway thanks for the good advice, and I totally agree that recording it right in the first place would be optimal ;) my problem is that the scratch track is out with the animation (for complex technical animation prototype reasons?*?*!), not much but just enough to be noticeable, doing an ADR session is not possible - so it is basically just watching the mouth and moving the audio to try to make them sound natural. I think it is just a word by word job ... – RedSonic01 Feb 14 '12 at 18:00
  • @RedSonic01 Ouch, sorry to hear that! I understand that sometimes re-recording dialogue isn't possible; just as long as your producers understand that the quality will be compromised and that, in the professional world, circumstances like these need to be prepared for. – Roger Middenway Feb 14 '12 at 19:08
3

You might have a speed problem. Is this PAL or NTSC? If it's too fast try resampling at 46034 Hz. If too slow, resample at 50000 Hz. Sync with plosives like P or B, also M's will work. You should not have to re-sync each word.

  • 1
    @william3 this is actually not too far from the actual problem, see the update in original post - thanks for the advice as it put us on the right track. – RedSonic01 Feb 17 '12 at 6:37
2

Frame by frame, line by line.....You have to accept the tedium of the task. That's the price for an inaccurate performance (if the sync was done after the picture).

Elastic Audio as well as some other TCE tools such as VocAlign will work...but I would cut the "lines" into the smallest possible pieces - using TCE as minimally as possible.

  • @Audiophile.2010 yep I think this is my fate - line by line ;) thanks for advice – RedSonic01 Feb 14 '12 at 18:02
2

B's and P's baby. To a lesser extent F's, M's and V's.

1

I have never had to attempt this, as most of the time the sync has been checked at the ADR stage or has been done properly by the animator.

From working with students who have recorded ADR that is a little out, I have asked them to look for Hard letters (technical term?); these give distinctive pops and good visual mouth references.

Rather than going for elastic audio, which still seems to colour the sound of dialogue IMO, I would prefer to edit bad dialogue over a couple of tracks and route to a group aux for better sync and overlap.

  • just realised someone has already answered with B and Ps.... dam – Nairn Beattie Feb 16 '12 at 15:51
1

In 15 years of editing and mixing animation, I've found that animators SELDOM get the lip sync exactly right, for a wealth of reasons, some of which the animators have little control over including time budget, and poor character designs. As far getting things to "feel right", it really boils down to just nudging things. I've moved lines of dialog 4 or 5 frames to make it feel right, even when things were "in sync" with the start of the audio. There are two main reasons.

One, animation uses only a limited number of "mouth positions"... anywhere from 2 (open and closed for very crude animation) to as many as twelve. Since the human mouth makes many more than that, often animators have to match as close as they can. Sounds like "s" can be difficult, since the mouth moves very litte. If you really pay attention to the way humans speak, you'll notice we can make a HUGE quantity of sound, while moving our lips very little.

The other reason is that animation is very often done "on twos". In other words, motion only happens every TWO frames.. hard consonants like a 't' can sometimes last only a frame, which makes things seem out.

Unfortunately, there's no hard and fast rule as to when to move forward, and when to move backward. A lot of my time doing Dialog sync in animation is spent moving things two frames this way, and then three frames that way until I find it "works". Having said that... most people notice dialog that's too early, before dialog that's too late (because we're used to sound travelling slower than light), so I usually start by nudging the dialog later... If that doesn't work, then I nudge it sooner.

  • @Sonsey - yes this is basically confirming my findings, I agree that it feels better if it is late, I also noticed that even a 1/4 of a frame can make a difference to perception of a word. but man you can get obsessed with the sync really quickly, far above what any normal person would notice. – RedSonic01 Feb 17 '12 at 7:03
  • Oh yes! And it will drive you NUTS when you watch Broadcast TV for a while! :) As you continue, you'll find that your instincts get better and you're able to get it right faster... at least for that particular show. – Sonsey Feb 18 '12 at 1:43
0

I do not agree! If the acting isn't close enough, it will sound and feel awful. Feel beats sync! When it's close already THEN you can align it. But if it's way out I'd go old school and match the end of the line.

0

Hi!

As someone said above, P's and B's are great points of sync. If the animation is close enough, you should place the very beginning of the "P" sound in the first frame the characters' lips are preparing for this letter. Vowels, most of all "A" and "O" are probably the best for stretching. I advise you, if you haven't went through yet, Jay Rose's Audio Post-Production for Film and Video.

Good luck.

-2

imho

There are only two rules:

1) Just begin to do it and most of your questions will disappear.

2) On editing (sync) stage your primary task is to sync the dialogue. Lips by lips, frame by frame and that's it. Later on re-recording stage re-recording mixer will do it so that it will feel right.

  • 3
    Sorry thats me With the -1... Not true at all. There's nothing I can do in the mix to add feel to a line of dialogue. It's got to be in the line. Getting it to mat h the surroundings is another matter entirely. If you cut it 100% tight it will likely sound awful! Use your ears and eyes, and when they don't agree go with gut and feel. – ErikG Feb 14 '12 at 17:43
  • Thank you Erik! I guess I had to write my opinion more correctly. – Conant Feb 15 '12 at 5:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.