HAP Codec

Suggestions and comments for Pandoras Box V5

Moderator: Moderator Group

HAP Codec

Postby florian » Sun Oct 11, 2015 3:39 am

Dear Coolux,

Please dedicate support/development resources to supporting the Hap codec family.

The attached image shows the results of my AE render Queue. The same 2048x931px 24fps 00:04:47:13 file transcoded from ProRes4444 to Hap and Clx_DDS_uncompressed.

On my 27" iMac, encoding to Hap takes about 18 minutes. The same encode to the coolux codec takes 1 hour and 18 minutes.

From my understanding (which is not complete, but fairly versed and extensive), these 2 codecs are basically doing the same thing, and have nearly identical quality profiles. I did a difference matte test and scrubbed through the results to be sure.

MPEG2/MXL doesn't hold up for quality to either of these two codecs, and for large format content, the DXT-based image formats are increasingly necessary to support quality requirements. On a recent project, we wound up having to let deliverables cook overnight to review them in the morning. An attempt to use the image converter and spreading the work across our playback machines, was abandoned when the image converter kept crashing. In the ideal world we no longer need to transcode deliverables. Putting the onus of delivering media for playback entirely on the content producers leaves us programmers to do other things (like sleep).

So for the sleepless nights on this most recent gig, this is a big *bump*. Please allow us to support Hi Quality Media formats that content producers can deliver without bringing their render pipeline to a halt. There are about half a dozen separate little utilities and gizmos packaged into the coolux installer and I really just want to let people deliver me media that I can play without having to touch any of them. That's how it works everywhere else I wind up working these days.

I would encourage users who need to invest valuable time in devising elaborate content ingestion workflows to chime in. Based on the success of the Lock-To-Time without Loop thread maybe, we'll have Hap support in 2017...

Thank you.
You do not have the required permissions to view the files attached to this post.
Last edited by florian on Sun Oct 11, 2015 4:16 am, edited 1 time in total.
-flo

"Wise men learn more from fools than fools from the wise."

Cato the Elder, Plutarch's Life of Cato
User avatar
florian
 
Posts: 154
Joined: Fri Jul 25, 2008 3:49 pm
Location: New York, NY

Re: HAP Codec

Postby mkohler » Sun Oct 11, 2015 3:56 am

+1. After attempting the CLX DDS workflow for 2 shows I ended up completely infuriated and having to switch to other formats at the last minute with costly transcoding time and embarrassing myself in front of multiple clients after explaining to them that this delivery format was supposed to be better.

The tools to convert into DDS/Clx are complete and utter garbage, with astronomical render times, dysfunctional GUI's, embarrassing stability, and totally inexplicable behavior across different systems. It isn't even supported in the pandora's box encoder extension, its just another crazy bolt on program. This isnt the kind of quality and reliability I have come to know from coolux products, and I know I'm not the only person who feels this way.

As Florian points out, the HAP Codec provides the same quality and performance profiles as CLX/DDS and has an active development cycle that is actually improving the code. The Quicktime plugins work well, are user friendly enough for artists to work with, and is quickly becoming The Defacto Standard for large format content playback. I've personally recommended other servers for projects with large format content because of the trainwreck caused by attempting to use your codecs despite having an inventory of 2015 Server Pro's sitting on the shelf gather dust.

MK
You do not have the required permissions to view the files attached to this post.
mkohler
 
Posts: 15
Joined: Mon Nov 30, 2009 9:41 pm

Re: HAP Codec

Postby alexbright » Sun Oct 11, 2015 4:04 am

Hi Florian,

I'm VERY glad you brought this up.

The Hap encoder works well for all of our clients, as they are able to produce content in the final playback format, quickly and without dirstrupting their workflow. I rarely get any questions, they install it on their rendering computer, and it just works. However, I have yet to have a client successfully produce anything in the coolux codec. The the built in transcoder is quite unreliable, and seems to poorly utilize system resources.

It really streamlines the content workflow for the animators to be able to render in the final playback format, and hap is making this very possible. The fact that all of the major media servers have adopted this has made it an industry standard in the last 12 months.

The last job I had spec’d to be Pandora’s box job (last month) ended up getting changed to another media server purely because of this issue. We had a 12896 x 1080 total content template, and we simply couldn’t encode to the coolux codec fast enough.

Please please consider adopting this industry standard so we can continue to utilize Pandora’s Box as well as we utilize it’s competing platforms.
alexbright
 
Posts: 12
Joined: Mon Aug 30, 2010 9:37 pm

Re: HAP Codec

Postby alexbright » Sun Oct 11, 2015 4:15 am

Oh, and Michael brings up an excellent point.

WHY does the encoder extension NOT support CLX DDS??? It really makes both the encoder extension as well as the PB Quicktime Converter both feel like a bolted on inconsistent parts of the UI.

I’ve long thought this about the encoder extension, but grudgingly accepted it as something that quickly makes Mpeg2… The fact that it won’t even make a coolux proprietary codec just makes it feel broken.
alexbright
 
Posts: 12
Joined: Mon Aug 30, 2010 9:37 pm

Re: HAP Codec

Postby aquapixs » Sun Oct 11, 2015 9:26 am

Hello there,

So another content discussion..... :-)

Funny enough I'm right now sitting on a show in the UK and this show is entirely running HAP on Pandoras Box.
And as far as now it works absolutely perfect. No issues at all....

I just downloaded the free HAP codec from D3s website and installed it on all PB machines. No issues at all.

But regarding quality it follows the same ways like DDS, so colour bending is visible especially on computer animated gradients. Real footage works very good.
I do have the feeling that HAP is a bit better looking than DDS.

In any caaes the full workflow and file handling is much better and fastest. No image Sequences with massive amount of files. Simple export thru Quicktime. Embedded audio if needed.

Still neither DDS nor HAP is getting somewhat close to uncompressed content. But so far HAP is the best alternative if you do not need uncompressed content.

I also made a lot of great experiences with BMP.mov snappy out of the Coolux encoder, which by the also renders into HAP if the codec is installed.
BMP is perfect in terms of quality but file sizes get crazy. I recently hat 1.5 TB of content for a 8 min show, all set in 50fps. So you need really big SSD harddrives and a smart content management. In that workflow I also do recommend 10 gig network.

So for me as of today I do have two choices for codecs:

High end quality / best colour quality / uncompressed workflow:

BMP.mov (in snappy) or BMP sequence in snappy
both possible with or without alpha

Hardware requirements for this:
- SSD drives with min 2.8 TB Raid
- high performance CPU (Intel Xenon E5 / I7 or similar
- 16GB Ram
- 10 gig network
- Nvidia GPU
Unfortunately Coolux has no hardware option close to that, so in an uncompressed workflow I usually end up with Player Pro and custom hardware.

All other shows:

HAP or HAP running on any hardware.
Generally I do recommend to use Coolux R3 Servers with SSD drives. Everything else is more or less outdated and not really working on high performance shows.

Let me know if any questions raise regarding this. I will know much more about uncompressed playback by the end of the year.

@Alex: I do totally agree that the Encoder Extension needs to be replaced or reworked with the new Quicktime component and needs a lot of other improvements.
Right now I´m running into the problem on about a 100 clips in HAP codec I have embedded audio which I need to extract to wav to run on Audio tracks. But doing this thru the
encoder extension will also convert the mov into m2v and replaces it in the project. so for each file I need to read the mov and drag it in the timeline again. Really very annoying.....

Best wishes

Patrick
Patrick Verhey
Director Media/Creative
______________________________________________________
PRG XL Video
A Division of Production Resource Group LLC

Mobile: +44 7702 878320
Email: pverhey@prg.com

2 Eastman Way, Hemel Hempstead, HP2 7DU
User avatar
aquapixs
 
Posts: 43
Joined: Mon Sep 29, 2014 12:00 pm

Re: HAP Codec

Postby cyeary » Sun Oct 11, 2015 8:29 pm

So I will also make a vote here that the DDS workflow is very troublesome and I have found little constancy to making it work. I have had content producers not be abel to install the plugin, Deliver files that were not interpreted properly by PB once they were imported, and the horrible "not enough time to render this" These are all major issues when you are dealing with stressful projects and time crunching media delivery. The idea of HAP is a good one because it is a proven file format. I am not sure of the performance trade offs but I know for the big large delivery pieces that come last minute it would be great to just put it in the machine and play it instead of having to stress about the transcoding process.
Cameron Yeary
Unlimited Visibility, Inc.
cyeary
 
Posts: 138
Joined: Sat Feb 04, 2012 11:14 pm
Location: NYC

Re: HAP Codec

Postby aquapixs » Sun Oct 11, 2015 10:18 pm

Hi Cameron,

how are you? Had been a long time did chat....

I did not do any real performance test yet, but here is the setup I run at the moment:

Server Pro Rev 3 Hardware with SSD drives, average manager with unknown specs, but with SSD:

Server 1: 2 x 1920x1080@60 / content size 800x640@30fps
Server 2: 2 x 1920x1080@60 / content size 1000x640@30fps
Server 3: 2 x 1920@1080@60 / content size 1200x640@30fps
Server 4: 2 x 1920x1080@60 / content size 1000x640@30fps

Due to the way the show runs each server renders a total of 4 video files at the same time.
Manager previews all servers.
Besides a few glitches in the manager preview while fading from one set to the other, all runs absolutely smooth and
without any issues. The manager is a bit on its limit with the preview, but it also runs 8 tracks of audio for each
video. So already a lot of work for him.

On the same manager I did also see two HAP files with 1920@1080@50fps on the preview at the same time.

So from my feeling at the moment, HAP pretty much follows the performance specs of DDS.mov

I might find some time tomorrow to run multiple HAPs with 1920x1080@50fps tomorrow during the day. But as we have a full rehearsal day I cannot promise anything.

best wishes

Patrick
Patrick Verhey
Director Media/Creative
______________________________________________________
PRG XL Video
A Division of Production Resource Group LLC

Mobile: +44 7702 878320
Email: pverhey@prg.com

2 Eastman Way, Hemel Hempstead, HP2 7DU
User avatar
aquapixs
 
Posts: 43
Joined: Mon Sep 29, 2014 12:00 pm

Re: HAP Codec

Postby florian » Sun Oct 11, 2015 11:28 pm

Hi Patrick,

You probably know this, but the place to properly download Hap's QT installer is from the project's homepage:
https://github.com/Vidvox/hap-qt-codec

I wouldn't recommend using a third party's links for it.
-flo

"Wise men learn more from fools than fools from the wise."

Cato the Elder, Plutarch's Life of Cato
User avatar
florian
 
Posts: 154
Joined: Fri Jul 25, 2008 3:49 pm
Location: New York, NY

Re: HAP Codec

Postby cyeary » Sun Oct 11, 2015 11:51 pm

Not bad Patrick however I have reliably ran 4 layers crossfading on R3 Machines with DDS. 6000x1080 But of course there are always trade offs.

Good to know that you have it working though.
Cameron Yeary
Unlimited Visibility, Inc.
cyeary
 
Posts: 138
Joined: Sat Feb 04, 2012 11:14 pm
Location: NYC

Re: HAP Codec

Postby aquapixs » Mon Oct 12, 2015 6:29 am

Flo:
You are absolutely right, your link goes straight to the codec. I did check the file you can download and it is technically the
same one.

Cameron:
I do belive that HAP will end up with the same performance as DDS, so I assume that you might be able to play the same amount of files on HAP as you can on DDS. As of today I can only tell for sure what I have used right now. There need to need to be some perfomance testing done. As Coolux does not officially support HAP its undortunately at the user side to do these tests. As this whole thinh is very important for some
upcomimg shows I might be able to do some propper test in the next week.
As soon as I know more I will post it here.

All the best

Patrick
Patrick Verhey
Director Media/Creative
______________________________________________________
PRG XL Video
A Division of Production Resource Group LLC

Mobile: +44 7702 878320
Email: pverhey@prg.com

2 Eastman Way, Hemel Hempstead, HP2 7DU
User avatar
aquapixs
 
Posts: 43
Joined: Mon Sep 29, 2014 12:00 pm

Re: HAP Codec

Postby florian » Mon Oct 12, 2015 2:02 pm

As far as performance goes, Michael pointed out (elsewhere) that you don't get full playback performance just by installing the QuickTime plugin. That just lets you play the files via QuickTime.

In order to get native GPU-based playback support, the coolux devs need to code in support for the format.

see 3rd paragraph:
https://github.com/Vidvox/hap-qt-codec/blob/master/README.md
-flo

"Wise men learn more from fools than fools from the wise."

Cato the Elder, Plutarch's Life of Cato
User avatar
florian
 
Posts: 154
Joined: Fri Jul 25, 2008 3:49 pm
Location: New York, NY

Re: HAP Codec

Postby aquapixs » Mon Oct 12, 2015 2:17 pm

Hasen't that been changed in 12086 with the new Quicktime codec implementation done?

That would be something someone in Coolux would need to tell....

Cheers

Patrick
Patrick Verhey
Director Media/Creative
______________________________________________________
PRG XL Video
A Division of Production Resource Group LLC

Mobile: +44 7702 878320
Email: pverhey@prg.com

2 Eastman Way, Hemel Hempstead, HP2 7DU
User avatar
aquapixs
 
Posts: 43
Joined: Mon Sep 29, 2014 12:00 pm

Re: HAP Codec

Postby florian » Mon Oct 12, 2015 2:43 pm

No,
for Hap playback performance to take full advantage of the codec you don't want to decode it via quicktime. Quicktime isn't built for multi-node playback. The container format is parseable, but you want the actual decoder in the system to speak Hap natively.
-flo

"Wise men learn more from fools than fools from the wise."

Cato the Elder, Plutarch's Life of Cato
User avatar
florian
 
Posts: 154
Joined: Fri Jul 25, 2008 3:49 pm
Location: New York, NY

Re: HAP Codec

Postby plbrunet » Tue Oct 13, 2015 11:12 pm

Great Subject,

+1 on this! I completely agree with everyone. Dealing with bringing in large format content in Pandoras Box is always long and cumbersome. Even bringing content with Alpha is too time consuming and more complicated than it should be.

Hap, Hap Alpha and HapQ support would be very much appreciated.

After that, can we find a way to easily and quickly program pre-split content to be played over multiple machines. This includes dealing with FXs over multiple machines as well.
Pierre-Luc Brunet
Audio - Video - Show Control
http://plbrunet.com
https://ca.linkedin.com/in/pierrelucbrunet
User avatar
plbrunet
 
Posts: 57
Joined: Fri Mar 13, 2009 11:49 pm
Location: Montréal, Canada

Re: HAP Codec

Postby florian » Thu Oct 15, 2015 3:02 pm

Hi coolux!

Any thoughts on this? We've been talking about this here for a week now and I'm guessing I'm not the only on who'd love to get some feedback from you about how likely this is to happen.
-flo

"Wise men learn more from fools than fools from the wise."

Cato the Elder, Plutarch's Life of Cato
User avatar
florian
 
Posts: 154
Joined: Fri Jul 25, 2008 3:49 pm
Location: New York, NY

Next

Return to Feature Wishlist V5

Who is online

Users browsing this forum: No registered users and 0 guests