Download the original attachment

OpenAFS Monthly Newsletter – August 2007

A Monthly Update to the OpenAFS Community compiled by the Advisory Council

Introduction

 

This is the first of what is intended to be a monthly communiqué from the OpenAFS Elders to the community of OpenAFS administrators and end users.  In these newsletters the Elders will summarize the activities of the OpenAFS community, the progress that has been achieved in OpenAFS development, and outline the road map of future development that could be accomplished with the availability of sufficient resources. It is hoped that these newsletters will strengthen the community and foster additional dialogues and participation.

Advisory Council Meeting

 

The OpenAFS Advisory Council met on June 26.  The following topics were discussed in the meeting:  Development, AFS & Kerberos Best Practices Workshop, Web Site Design, and OpenAFS communications.  Minutes of the meeting can be found at:  http://www.openafs.org/eldershttp://www.openafs.org/elders/JA

Recent Releases

Product ReleasesJA

UNIX, Linux & MacOS X Development

 

The following have been areas of recent activity for OpenAFS on UNIX and MacOS X: 

Microsoft Windows Development

 

We have issued five releases of OpenAFS for Windows in the past three months: 

AFS & Kerberos Best Practices Workshop

 

The AFS & Kerberos Best Practices Workshop was held at SLAC from May 7-11.  The keynote was entitled "AFS Best Practices in the Business Context" and was given by Mike Potek of Pictage, Inc.  Mike provided a compelling story of AFS usage within a global business whose distributed storage is growing at a very rapid rate.  

Most of the presentations (including the keynote presentation) from the workshop can be found at:   http://www.pmw.org/afsbpw07/workshop.html. The "OpenAFS for Windows and Kerberos for Windows User Interface Futures" and "OpenAFS Road Map

Discussions" can be found at:   http://www.secure-endpoints.com/talks/KFW_OAFW_Futures_BPW_2007.pdf and http://www.secure-endpoints.com/talks/openafs_roadmap_BPW_2007.pdf

Web Site Design

 

The OpenAFS Elders have a machine up and running at Stanford which is intended to be used for hosting a new customer focused web site. There is still much work to do on content design.  Sine Nomine Associates has offered to volunteer a graphic artist / web designer.  However, before this resource can be effectively used, the Advisory Council needs to develop the site content, navigation design and site mapping. JA

Documentation Improvement

There was discussion of the need for improvement of the OpenAFS documentation during the Best Practices Workshop. Thanks to Simon Wilkinson, the OpenAFS documentation is improving.  However, the manuals still need quite a lot of cleanup.  If anyone is interested in helping with this project, please contact Russ Allbery.

Case Study of the Month – Pictage, Inc. 

http://www.pictage.com/

Pictage is a worldwide provider of services to wedding photographers who upload the photos they have taken. Pictage then makes the photos available to the wedding party for photo selection and printing.  Photos selected for printing are retouched by hand using Adobe Photoshop.  Pictage uses OpenAFS to store all of the original photos, the processed photos, and the web content.  Work flow is managed by the use of mount points and symlinks which are placed within directories assigned to various photographers and their associated events. 

Pictage storage has exceeded 200TB of data and more then 40 million files.  They frequently push the limits of the AFS directory structures running into the directory entry limitations.  Directory search performance is one of their bottlenecks.  Unlike more modern file systems, AFS does not use a B+ tree representation and directory searches within the case-insensitive Microsoft Windows client are linear in nature.  This wastes significant CPU utilization and clock time.  Pictage had measured the performance loss on a duo-core Xeon processor at between 3 and 5 seconds to open a file in Adobe Photoshop when the directory contains 15,000 entries, because Adobe Photoshop will request information forthe last 30 files it has seen, thus requiring heavy usage of AFS directories.   OpenAFS for Windows 1.5.13 added an optimization for cached files with registered callbacks that avoids the performance problem for files that are known to exist.  With this optimization in place, Pictage experienced file open times similar to those experienced when opening files on the local filesystem.  For files that do not exist such as Java class files, the optimization does not apply, and the linear search must still be used. 

Number of simultaneous users: 150 AFS client users; 1000 web server users 
Amount of data in AFS: Approx 215TB 
Amount of AFS storage: 265TB with planned growth to 425TB in twelve months 
Number of volumes: Approx 800,000. 
Number of files: Approx 200 000 000. 
Number of file servers: 70 with 10 more planned 
Approx storage per server: depending on age of server 1.2TB to 25TB

(statistics as of 22 May 2007)