Whatever new asset management system you are going to use, you surely want to migrate your existing data and metadata from FCSvr to the new system.

If you need help extracting these data from Final Cut Server, please feel free to get in touch with me. I can help generating XML import files for your new DAM as well as reusing existing proxy files.

If this sounds interesting to you, please use my contact form to discuss your options.

In short, I would say, that

• Apple opens up its pro software like FCP X and Mac OS X Server to the consumer, by making it easier to learn than ever before. A great strategy to sell more boxes and devices, which surely leads to greater turnover, and probably to more profit, too, as consumers buy easy tools and need support on a very low level only (if at all), which Apple can provide for cheap.
• Apple dumps all network products, which’s sole purpose was to help sell more client boxes to professionals in the enterprise or workgroup, like Xserves, Xserve RAIDS, Final Cut Server, and probably -on the long run- other products like Xsan metadata controllers and Mac OS X Server. In fact, I don’t think there is a big need for any of these products anymore, since Apple clients integrate just fine with Active Directory environments, most file servers, and many storage devices.
• I don’t think that this necessarily means Apple doesn’t care for the pros anymore. I think that FCP X is already quite powerful, and multicam support, FCP X APIs/XML interfaces, SDI connectivity and the like are going to be added to FCP sooner or later. Yet, I think Apple’s main market for FCP X is the single user or small workgroup. If you look at FCP X’s media import options, you’ll see the option “Create proxy media”: If you also look at the Sharing options: you’ll see that FCP X includes an option to export files and probably metadata to CNN iReport. I bet that FCP X’s powerful metadata management capabilities and the option to generate proxies during media import, are a very good basis to add iCloud support to FCP X. I guess that for many very small workgroups it would be great to simply send a proxy version of a project to the iCloud, so that another team member can edit a sequence and send it back to the original FCP X machine which has access to hires files to render the final output. This might be a very good Final Cut Server replacement for very small teams.

Making tools like FCP simpler than before doesn’t necessarily mean, that it’s not suited for pros anymore. Okay, right now it is not, but I am sure, it will soon be.

Yes, Apple has dumped Final Cut Server, which is the base for many postpro environments, now. But FCSvr meant to most of its customers, that you needed Xsan in place (most resellers and consultants were simply not able to design these systems right), that Open or Active Directory needed to work fine and that your Mac and Windows clients needed to be integrated into this directory infrastructure (and again, most service providers could not do these things properly).

This means it was very difficult to get a Final Cut Server infrastructure working, especially as so few people know how to make storage, asset management, Open or Active Directory, filesystem permissions, portable home directories and video editing clients work together smoothly.

In my opinion for many small shops it would be a great step to cut out all this overhead and move FCSvr into the cloud by adding an option to FCP X.

On the other hand, FCSvr leaves a big gap in the market, which tools like CatDV might fill: many shops do not want to publish their data into the cloud.
Especially in Europe people don’t like to store their business data on servers hosted either in the US or anywhere else by a US company, which might sooner or later grant access to these data to any kind of secret service.
This leaves a need for a local DAM in enterprise environments.

Personally, I would like to have a much easier DAM tool than anything on the market, something that can be set up without needing a consultant, at least for small environments. For bigger environments, I would say that if you work with FCSvr, you will be able to keep your current infrastructure and just replace either the FCSvr server software or probably the whole FCSvr machine to get something else up and running.

As FCSvr is so open, it is totally easy to export media and metadata to any other system.

I had a look at the new FCP X project files and the way FCP X stores files on a central storage. Even without the Apple API I think that there is a way to extract data easily and build them into your own asset management.

Being a consultant, my approach to Apple’s latest announcements is this:

• I recommend people to use Windows servers for directory management.
• Don’t know if Xsan will really be free in Lion. Still Xsan is a great tool, but I might recommend people to use StorNext metadata controllers at least if people need rack-mountable hardware.
• Apple builds wonderful standalone machines, which can be integrated into enterprise environments. In fact, now that Apple focuses on its core business – building great machines which are easy to use – and doesn’t care much about the enterprise network stuff around it, there’s more money in the game for third party developers who build DAM systems. I bet we’ll see great third party solutions soon and overall Apple kit will better fit into enterprise environments than today.

I feel that Apple’s approach on the pro market might leave us all with better tools in the end.
Let’s see.

Maybe you already know fiprecon, my little middleware between Final Cut Server and Archiware PresSTORE, which allows Final Cut Server to archive media on tape.
In this article I’ll explain how to install an extended version of fiprecon, which adds the following features to the basic version:

• recursive archiving and restoring of productions and FCP projects
• stores id, label and barcode of the tape on which an asset has been archived in the asset’s FCSvr metadata
• sends a mail to an administrator, if a tape needed for a restore process is not available. Includes a list of missing tapes, supports authorized mail sending

Please make sure you try this in your lab first, to make sure everything works as expected.

Please download this zip file, copy it to your computer and extract it.

Login to the Terminal using the same user credentials you used to install Final Cut Server.
Now create /usr/local/fiprecon:
mkdir -p /usr/local/fiprecon
Copy the contents of the fiprecon folder into it.

Install the launchd plist files into /Library/LaunchDaemons and set the permissions to
-rw-r--r-- 1 root wheel 429 May 4 2010 de.andre-aulich.aw2fcsvr.plist
-rw-r--r-- 1 root wheel 429 May 4 2010 de.andre-aulich.fcsvr2aw.plist
Launch them:
sudo launchctl load -w /Library/LaunchDaemons/de.andre-aulich.*
Now run the config tool:
sudo /usr/local/fiprecon/bin/config.command

Add this line to your /etc/sudoers file:
ALL ALL = NOPASSWD: /Library/Application\ Support/Final\ Cut\ Server/Final\ Cut\ Server.bundle/Contents/MacOS/fcsvr_client

Add this line to your /etc/newsyslog.conf file:
/usr/local/fiprecon/logs/activity.log 777 10 100000 * J

Now open your FCSvr admin panel and create these items:
Metadata fields:
Unicode Strings:

Boolean:

Category is always “None”.

Metadata groups:





Please add the fields “Send FCP Project To Archive” and “Restore FCP project from Archive” to the Asset Filter MD group:

Please add the fields “Send Production To Archive” and “Restore Production from Archive” to the Production Filter MD group:

Responses:

Command path is /usr/local/fiprecon/bin/archivefcpproject.command
Command Parameter is [Asset ID]

Command path is /usr/local/fiprecon/bin/archiveproduction.command
Command Parameter is [Production ID]

Command path is /usr/local/fiprecon/bin/restorefcpproject.command
Command Parameter is [Asset ID]

Command path is /usr/local/fiprecon/bin/restoreproduction.command
Command Parameter is [Production ID]

Subscriptions:





Devices:
Add /usr/local/fiprecon/bin/postarchive.command as a post-archive command to your archive device.
Add /usr/local/fiprecon/bin/prerestore.command as pre-restore command.

Now fiprecon should work like this:
1. You can still select one or more assets in the FCSvr client interface and send them to the archive.
2. Instead, you can also select an FCP project asset and archive/restore entire FCP projects:

3. The same works with productions:

4. After PresSTORE wrote a file to tape, fiprecon will write the tape id back to the item’s FCSvr metadata:

If you archive and restore the same asset multiple times, fiprecon pro will only store the tape ID of the latest archive process, as a normal restore process will always access the latest version only.
I added some functionality to send authenticated emails if tapes needed for a restore process are missing. You will get the idea when you run the config tool.

Enjoy!

As long as you work with FCP 7 and not FCP X, you can still use FCSvr as one of the best DAMs/MAMs available for FCP environments.

If you start from scratch and want to build a new DAM environment, though, CatDV is a good alternative to Final Cut Server.
One of its greatest features is that it’s still available. Just joking, it has always been a great product.

Same as FCSvr, CatDV can not archive media to tape directly.

Mike Szumlinski wrote a very nice article including some scripts, which allow you to connect CatDV to Archiware PresSTORE.

Follow his guide, and it will be very easy to use CatDV as the frontend for your tape archive.

Since Apple has discontinued the Apple Xserve in January 2011, there seems to be a growing need for rack mountable server hardware in the Apple world, which is more powerful than the Mac mini Server and takes less rack space and consumes less power than the Mac Pro.

If you want to run file services, directory services, software update services, and other services for Macs, Windows, and Linux clients, you’ll find both Linux and Windows based solutions to do just that. On the other hand, the Mac Pro and the Mac mini Server seem to be great systems to serve many of these things, too.

Yet, if you read the announcement of ActiveSAN, it seems as if there was a high demand for a 1U rack mountable metadata controller for Xsan environments.
What the guys at ActiveStorage seem to do is build great server hardware incl. nice monitoring and setup tools, which make it easy to set up a Quantum StorNext metadata controller to handle your Mac-based Xsan clients.

If you are an Xsan or StorNext pro anyway and work in the command line most of the day, you can even replace your existing Apple Xserve MDC with a Linux-based StorNext MDC with a minimal downtime of your Xsan volume. This is what you need:

• Server hardware which is able to run a recent Linux version. Make sure you add enough RAM, redundant drives for the boot disk, a nice and OS supported FC card, dual Gigabit Ethernet. Note: You can also use a Windows OS instead, which is nice if you integrate with Active Directory, but whatever you do, make sure, the OS of choice supports the LUN size you use in your Xsan setup. The ActiveSAN hardware looks like a very nice solution to this.
• I would go for CentOS as the MDC’s OS, as it’s officially supported by Quantum and doesn’t cost you money. If you need support, I would go for RedHat. See http://downloads.quantum.com/SNMS/4.1/StorNext%204.1%20Supported%20Platforms_V2.pdf for a list of supported MDC platforms.
• Make sure to integrate your MDC into your directory environment. Both Active Directory and Open Directory are supported. You will need this to use ACLs (the only way to properly deal with permissions in most Xsan environments).

Before you start buying the hardware to build the new MDC, you need to make sure, that you don’t have named streams enabled in your existing Xsan environment.

To actually migrate the volume from the Xsan MDC to the SNFS MDC, roughly follow these steps:

• Make sure to use matching filesystem versions in the Xsan environment and on your StorNext controllers.
• Run cvfsck -w on the Xsan volumes.
• On the primary Xsan metadata controller, save the folder /Library/Filesystems/Xsan/config to a place we can access easily later on, like a USB stick, e.g.
• Shut down the Xsan metadata controllers.
• Set up the StorNext controllers. The easiest way is to use the same IP addresses as for the Xsan metadata controllers.
• After installing the StorNext filesystem and -optionally- the Storage Manager, copy our Xsan volume’s config file into the StorNext controller’s config folder.
• Open the config file and delete the lines with the options “NamedStreams”, “EnableSpotlight”, and “SpotlightSearchLevel”.
• Optionally, make the volume managed by adding the appropriate option to its config file.
• Make sure to run chown www:adic on the config file as well as chmod 664
• Run cvfsck on the volume.
• Start the volume and mount it.
• You are done.

If you want to add Xsan clients to your StorNext controller, you find a nice article at http://krypted.com/xsan/adding-xsan-clients-to-stornext-environments/.

While this migration is something that can be done very easily, you need to consider, that StorNext currently doesn’t support Spotlight searches, so you can’t easily use the Finder to search for content. If your users navigate through the filesystem or use a DAM like Final Cut Server, this is not an issue at all. (On the other hand, the StorNext MDC catches all filesystem events, so I guess it should be possible to write a client, which collects these events and tells a dedicated Mac machine to add new files to the filesystem’s central Spotlight DB).

Another caveat is that the automatic HSM process in Storage Manager, which moves data to tape and restores them to disk as needed, restores data whenever a file gets invoked by a client. You either need to turn off previews in Finder to prevent constant restores, or you better hide the filesystem from the Finder and let your users access the filesystem through FCSvr and its edit-in-place option only.

One of the best things that come with the migration from Xsan to StorNext is that you can add additional functions (part of SNFS and/or SNSM) like filesystem replication, Distributed Data Mover, HSM functionality, writing to tape, etc.

In addition, SNFS has documented optimization settings for image sequences, which is very interesting for film production.

On the other hand, if you already own an Xserve which runs your Xsan, I would say that this is a great hardware to do just that.

Although I have done the migration described here in the lab, and found it reasonably easy, I am curious to see ActiveSAN as soon as it’s there, because it looks like they made the process of building a new Xsan/StorNext environment so much easier.

If e.g. a media file has the file path /Volumes/MyBigStorage/MigratedFiles/cars/german/bmw/Z4_in_summer_with_blue_sky.jpg, then both the file path elements and the file name hold valuable asset metadata which you might want to use in your FCSvr DB.
If your FCSvr device has the file path /Volumes/MyBigStorage/MigratedFiles, then the relative path (called Location) stored in the asset’s metadata will be /cars/german/bmw, while the file name will be Z4_in_summer_with_blue_sky.jpg.

If now you search for

cars bmw

FCSvr won’t find this asset. You need to search for cars/german/bmw to find this asset, as FCSvr doesn’t support finding a single string when you search for multiple parts of it.

Your users expect to find the asset using the search string “cars bmw”, so we need to fix this. This article explains how to do this.

PLEASE NOTE: If you want to mirror your filesystem within Final Cut Server to help your users find their content using the same folder structure they are used to from the filesystem, please get in touch with me. I’ve got scripts which scan the filesystem and tell FCSvr to create folders and subfolders as productions and subproductions to do exactly this.

So what are we going to do?
Whenever a file is added to the FCSvr database, we want FCSvr to take the location and the filename, extract all valuable metadata as individual strings and add them to the metadata field “Keywords”. In our example, we would translate “/Volumes/MyBigStorage/MigratedFiles/cars/german/bmw/Z4_in_summer_with_blue_sky.jpg” into “cars german bmw Z4 in summer with blue sky jpg” and add this string to the Keywords field. We need to make sure that we don’t overwrite existing keywords.

To do this, please follow these steps:

1. On the FCSvr machine, open the Terminal and get root permissions. Now type:
   mkdir -p /usr/local/path2key/bin
mkdir -p /usr/local/path2key/logs
cd /usr/local/path2key/bin
touch path2key
chmod +x path2key
touch ../logs/path2key.log
chmod 777 ../logs/path2key.log
2. Now copy this text into /usr/local/path2key/bin/path:
   #!/bin/bash
   #
   ###################################################
   # path2key, copyright 2011 by www.andre-aulich.de #
   # This script accepts the asset id of a FCSvr #
   # asset as its only parameter. It then extracts #
   # the file path and adds all parts of the path as #
   # keywords to the asset's FCSvr metadata. #
   ###################################################

   # Define our variables and set up logging.
   PATH="/Library/Application Support/Final Cut Server/Final Cut Server.bundle/Contents/MacOS/":$PATH
   LOGFILE="/usr/local/path2key/logs/path2key.log"
   ASSETID="$1"
   echo "$(date)" >> "${LOGFILE}"
   if [[ "$1" == "" ]]; then
   echo "No asset ID given. Stopping." >> "${LOGFILE}"
   echo "---------------------------------------------------------" >> "${LOGFILE}"
   exit 0
   fi

   echo "Processing file path of asset ${ASSETID}." >> "${LOGFILE}"

   PATHKEYWORDS="$(fcsvr_client getmd /asset/${ASSETID} | grep "PA_MD_CUST_FILENAME:\|CUST_LOCATION:" | sed -e 's/CUST_LOCATION://' -e 's/PA_MD_CUST_FILENAME://' -e 's/^[ \t]*//' -e 's/^\///' -e 's/[\.\\\/-]/ /g' -e 's/_/ /g' | awk '{printf("%s", $0 (NR==1 ? " " : " "))}')"

   echo "Keywords extracted from file path are:" >> "${LOGFILE}"
   echo "${PATHKEYWORDS}" >> "${LOGFILE}"

   FCSVRKEYWORDS="$(fcsvr_client getmd /asset/${ASSETID} | grep "CUST_KEYWORDS" | sed -e 's/CUST_KEYWORDS://' -e 's/^[ \t]*//')"

   echo "Keywords extracted from FCSvr are:" >> "${LOGFILE}"
   echo "${FCSVRKEYWORDS}" >> "${LOGFILE}"

   COMPLETEKEYWORDS="$(echo ${FCSVRKEYWORDS} ${PATHKEYWORDS})"

   echo "Complete keywords are:" >> "${LOGFILE}"
   echo "${COMPLETEKEYWORDS}" >> "${LOGFILE}"
   sudo fcsvr_client setmd /asset/"${ASSETID}" CUST_KEYWORDS="${COMPLETEKEYWORDS}" >> "${LOGFILE}"

   echo "---------------------------------------------------------" >> "${LOGFILE}"

   
   or download this file instead and extract it. This script will later extract the metadata from the file path and the filename. It replaces . / \ – _ – against whitespaces.

3. Now open the FCSvr Administration window and create a new response using these data:
   • Response Action: Run an external script or command
   • Name: path2key
   • Command path: /usr/local/path2key/bin/path2key
   • Command Parameters: [Asset ID]
4. Create a Subscription using these data:
   • Subscribe to: Asset
   • Name: path2key
   • Enabled: needs to be activated
   • Event Type Filter: Created
   • Response List: path2key
5. To make sure our script can write to the FCSvr database, we need to add this line to /etc/sudoers/:
   ALL ALL = NOPASSWD: /Library/Application\ Support/Final\ Cut\ Server/Final\ Cut\ Server.bundle/Contents/MacOS/fcsvr_client
6. You might need to restart the FCSvr client interface, and probably the FCSvr server, too.

If now you upload some files to FCSvr, please look at their Keywords metadata field. The file path and file name elements should be added to the list of keywords, and now your users will be able to search for

bmw cars

to find your asset.

Hope this helps.

If you need more customization like this, I’d be happy to provide my services and help you develop your environment.

This version includes the following changes:

• Better handling of files with file names including double spaces (thanks for the hint, Matt Geller)
• Automatically deals with non default PresSTORE Archive Indices (thanks to Lars Petter Ommundsen at Video 4)
• Better config tool handling
• updated manual to reflect fsevents support of PresSTORE 4

You can pick it up at http://www.andre-aulich.de/en/perm/connecting-final-cut-server-to-archiware-presstore.

I thought about adding these features, too:

• Preventing duplicates in the archive
• Writing tape ID back into an asset’s metadata in FCSvr
• Automatic creation of temporary Archive folder, responses, and metadata fields in FCSvr during config process

Yet, right now I think that these features need to be customized for each installation, so I decided not to add it to a standard tool like fiprecon.

If you need one of these features, please feel free to get in touch with me.

It totally makes sense to consider the camera preferences of your OB teams and the storage capacity your raw material is going to occupy in your post and in the archive, and these two points seem to be the main points influencing format decisions.

Yes, XDCAM HD 422 is a 50MBit format, and AVC Intra needs 100MBit for comparable image quality, that’s why many people think that XDCAM HD 422 is more effecient and therefore the format to go for.

Yet, XDCAM HD 422 is NOT a constant bit rate (CBR) format per frame, as it is a long GOP format, where an i-frame takes up to roughly 120 MBit, while the following frames go down to around 30 MBit. This means that XDCAM HD 422 is a variable bit rate (VBR) format with roughly 50Mbit/s as average.

In opposite, AVC Intra IS an i-frame only format and has a constant bit rate, which allows to have simpler header structures and make it more effective in postproduction.

As i-frame only formats are much more effective in the post, most XDCAM HD 422 houses convert their XDCAM HD 422 footage to ProRes or some other i-frame only format before they edit it. This brings up new problems: the material in the post is already the second generation of the clips, and you don’t want to send ProRes files into your archive, as it is not an SMPTE standard.

So probably you end up re-encoding your material to something else before you send it into your archive. This would at least be the third generation of your raw material. If you ever need to fetch this material from the archive and work on it again, quality will decrease with every new generation of your clips.

If instead you choose AVC Intra as your main production format, you can fetch it from the OB team and directly work on it in the post without re-encoding it.
You can even change the OP Atom structure to OP 1a (or vice versa) by simply rewrapping files without re-encoding, so you would still work on the first generation of your raw material.

AVC Intra is also more appropriate for the archive than ProRes is (though an open codec like in the famous BBC project would still be more preferable), so you might have an end-to-end workflow without ever generating more than a second generation of your material.

But now comes the really interesting part: Imagine you create two sequences in Final Cut Pro (or any other editor), one for XDCAM HD 422 material, one for AVC Intra material. If you only use hard cuts in your timeline, exporting the final clip from the AVC Intra project would simply copy all the i-frames from the source material into the new file and write a new header – no compression or re-encoding necessary.

If you do the same with your XDCAM HD 422 project, your editing application will need to re-encode parts of your sources, as your cuts will most probably be somewhere within long GOPs. As the first frame after a cut needs to be an i-frame, your application needs to calculate the content of the previously referencing frame and reference (and encode) the following frames of the GOP to the newly created i-frame. This way each cut can trigger a render operation for up to more than one second of source material. For each cut!

So even with hard cuts only you can generate a second generation clip with XDCAM HD 422 material, while AVC Intra material would still be first generation.

Though this should not be the only point to consider when you decide for your main production codec, I tend to think that choosing AVC Intra over XDCAM HD 422 can help you keeping image quality throughout your workflow high as you need to re-encode less.

If you want me to help you defining your end-to-end video (or film) production workflow, please feel free to get in touch with me.

Imagine a TV station airing multiple news formats as well as magazines every day. At some stage, a journalist and editor will output a 20 second clip for the news, containing both fresh first generation material and already compressed, 2nd, 3rd, 4th generation material from the archive.

If this clip needs to be used in some TV magazine or another news show later in the day, people might want to add or remove content to develop the story during the day. Without an efficient DAM, editors might tend to take the rendered clip from the morning, edit it and reencode it. This way the editor produces multiple generations of the file during the day, and in the evening of that day, the image quality of the clip will be poor.

The solution to this would be to link any created clip to its source files and the related FCP sequence. This way all the editors would access the original source files and would only generate a second generation of clips during their work. Depending on your wrapper format, the codec and the question if you use any other effects than hard cuts, you might even not generate a second generation at all. (This will be the topic of my next article).

So the big thing here is to link any ready-to-air clips to their source media and FCP sequences to maintain a high level of image quality.

If you want to know how Final Cut Server can help you keep image quality on a high level throughout your whole production workflow, please feel free to get in touch with me.

First of all, Windows 7 clients are not officially supported with Apple’s Final Cut Server. Yet, I have installed enough Windows 7 machines talking to FCSvr to say that this works smoothly.

And this is how you do it:

1. Make sure that your FCSvr machine has a unique DNS name on the AD servers. Both DNS lookup and reverse lookup should work reliably.
2. Bind the FCSvr machine to your AD using Directory Utility if it’s a 10.6 client. If it’s a 10.6 server, use Server Admin and then Directory Utility.
3. Do the same thing with all your Mac clients.
4. Now bind your Windows 7 machines to the 2008 AD.
5. And now the tricky part: For every user who needs to access Final Cut Server, activate Kerberos DES encrpytion in the AD: (Sorry, just got this screenshot in german, but I am optimistic, that you can easily find the correct setting on an english system, too.)
6. Now make FCSvr talk to the AD following this instruction.
7. You will also need to map AD groups to FCSvr internal groups called permission sets using System Preferences on the FCSvr machine.
8. Install the appropriate Java version on your Windows machines, then the latest QuickTime version. Then download the FCSvr client.
9. To try out if your configuration works fine, log into the FCSvr client from both Macs and Windows machines. These machines need to be bound to the same AD like the FCSvr server. While logging in from Macs should work fine using usernames in the form of “fcsvradmin”, on the PCs you need to use user names in the form of “fcsvradmin@MY_AD_DOMAIN.COM”, where the domain part of the login name has to be uppercase, so better you choose the long version for both Macs and PCs to make it consistent. Make sure to use the FCSvr server’s FQHN as the target for the FCSvr client app, too. The whole thing looks something like this:

I have seen situations where the Kerberos files on the FCSvr machines were messed up after extensive testing. If you need help fixing problems with FCSvr in AD environments, please feel free to get in touch with me.

I’d also appreciate to get your feedback if this article has helped you with your setup!

Good luck,

André