Trusted Operating Systems
BigAdmin recently ran a comparison article by Glenn Faden about Solaris Trusted Extensions and Red Hat Enterprise Linux 5 that I think deserves some response. Partially because of factual errors about SELinux in the article and partly because I think that there are some big picture misconceptions.
Mr. Faden starts off with:
“Sun Microsystems, Inc. and Red Hat have both submitted new versions of their trusted operating systems (OS) for Common Criteria (CC) certification evaluation. While these systems are being evaluated against the same CC protection profiles and at the same evaluation assurance level, these systems differ in significant ways that affect how a customer might choose to use such systems.”
The biggest misconception of this article that I want to address is that Red Hat Enterprise Linux 5 is a “trusted operating system”. It is not and hopefully never will be. Instead, Red Hat Enterprise Linux is a general purpose operating system that can meet the same requirements that traditionally required a special-purpose trusted operating system. This distinction may seem small, but it has large implications on the relevance and long-term viability of Red Hat Enterprise Linux and SELinux.
Trusted Operating Systems
In the past the major Unix vendors would take a version of their operating system, bolt on security extensions for multi-level security, and release it as a trusted operating system. This includes Trusted Solaris, Trusted IRIX, Trusted HP-UX, and others. By the time the vendors finished the modifications, shepherded the system through CC evaluation, and made a release it was normally at least a version out of date from the latest available general purpose version. In addition to being out of date the trusted systems often lacked third-party support and had numerous application incompatibilities.
It was also a huge pain for the vendor to maintain these separate releases. The trusted variants included complex modifications to the very lowest levels of the operating systems that often had a ripple effect throughout the entire operating system and related utilities. I’m certain that the goal was always to have as minimally invasive changes as possible, but the systems were always significantly different from their mainstream counterparts. Maintaining these changes, either as separate code bases or separate builds, was difficult and expensive.
The worst of it, though, is that these systems were only useful for a small set of customers. Namely, government customers that maintain classified data (you know, “Top Secret”, “Secret”, etc.). The systems weren’t super secure versions of their general purpose counterparts as many people mistakenly assume. Instead, they were only secure in terms of a very narrow set of requirements. The security added was aimed solely at maintaining the confidentiality of data organized as a set of hierarchical sensitivities with categories representing compartments. In other words, classified data that only certain government organizations needed to store and process on computers. The systems didn’t address general security problems or even the data confidentiality needs of the commercial world.
This meant that the huge expense and engineering effort was really only for one set of customers. Granted, these customers tended to have large budgets, but not compared to the overall commercial market. It is not surprising, therefore, that these trusted operating systems have been disappearing slowly over the years. They just aren’t economically viable. Sun is one of the last holdouts with Trusted Solaris 8, but Solaris with Trusted Extensions marks the end of that product.
Solaris Trusted Extensions is attempting to produce a system that meets the needs of government customers in a cheaper, more sustainable way. Their approach is to make minimally invasive changes to the operating system by leveraging a generic feature of Solaris: zones. The changes they are making are still aimed squarely at the government problem set; zones have some nice security properties on their own, but the changes made by the Trusted Extensions are only, as far as I can tell, applicable to containing classified data. The advantage for Sun is that the changes are much more localized than in the previous version of Trusted Solaris. This will allow them to invest less money and track general Solaris development much more closely.
The technical approach is interesting, even clever, and is probably effective at limiting the engineering cost of maintaining the trusted variant. Ultimately, though, what is created is a set of single level systems running on a single piece of hardware. Some of the features, like the X windows system, make the multiple systems behave as a single system, but the illusion is really skin deep. Applications are forced to communicate using standard networking and shared file systems even when local communication would have been allowed by the policy (for example, upgrading of data from Secret to Top Secret is generally allowed) [Update: Mr. Faden states in the comments that limited forms of inter-process communication are possible, though the general point still remains]. Additionally, I have questions about whether it will be possible to create enough zones to separate the number of levels on real multi-level systems (which can have tens-of-thousands of distinct levels), but that is a topic for another post.
Ultimately Solaris with Trusted Extensions is just a better way to create a trusted operating system and it still only meets the needs of government customers.
[Update: just to clarify: "better" here means that it seems that the current Trusted Extensions approach is a cheaper and more sustainable way to turn Solaris into a multi-level operating system. Not that the resulting product is better for the customer (it may or may not be, I think that is yet to be seen) and certainly not that it is a better way to create a multi-level system than SELinux. If it wasn't clear from the rest of this entry, I'll state it directly. I think SELinux is a better way to meet the security needs of a variety of customers, including government customers, than either the old Trusted Solaris or Solaris with Trusted Extensions.]
General Purpose Security
SELinux, and by association distributions that include it like Red Hat Enterprise Linux, has a very different set of goals than a traditional trusted operating system. SELinux aims to create a general purpose security mechanism capable of meeting a variety of requirements. It’s primary focus is on addressing the security problems plaguing general purpose operating systems. It also aims to address data confidentiality in a way that is acceptable to general users and government users. The same approach has been taken for the other security components in Linux (like the audit system).
There is no trusted variant of Red Hat Enterprise Linux. The system submitted for evaluation is just a special configuration of the general release. The goal is for all of the code to be shared (and this goal has largely been met – even the few last minute changes made for evaluation will be feed back into the general release as updates). The SELinux policy is slightly different, but only in an additive way. All of the general security improvements provided by SELinux are still available; there are just additional controls on data confidentiality.
In addition to the technical differences in the approach there is a difference in the practical approach. SELinux has been open source for several years and largely integrated into the upstream projects of Linux. Most notably, it is included in the mainstream 2.6 kernel. SELinux isn’t used by default in all distributions, but it is included in many including Gentoo, Debian, Fedora, and Red Hat Enterprise Linux. There is also an active and diverse set of contributors to SELinux and SELinux related projects from a variety of organizations (some of the largest corporate contributors are Red Hat, HP, IBM, Tresys Technology, and Trusted Computer Solutions, but there are many other organizations and individuals involved). It is a true open source project, not just a project with source code available.
This approach has resulted in the rapid evolution of SELinux. Integrating strong access control into a general purpose operating system is a challenging project. The community effort is meeting that challenge without compromising the goal of creating a generally useful security mechanism. This means that while we don’t currently have a multi-level desktop available, SELinux awareness is being integrated into the Xorg X windows system. It is being done as part of the larger X development community with cooperation and involvement from the upstream community (the basic component, Xace, has already been merged).
This community approach is scalable and sustainable in the long term. It also means that SELinux benefits from interaction with a diverse set of users and developers and those projects benefit from the access control approach taken by SELinux. By being a part of mainstream Linux development SELinux advances as Linux in general advances. The differences present today between Red Hat Enterprise Linux 5 and Solaris with Trusted Extensions are less important than the differences that will present in the coming years. The approach and development model of SELinux will ensure that it is more relevant and more generally useful than an approach aimed only at a small subset of users. This is good for everyone, even the government users.
Having said all of that I can’t stop without pointing out some of the largest technical errors in the article. I don’t want to do a point-by-point rebuttal, just touch on the highlights.
OS Integration
Both SELinux and Solaris Trusted Extensions include changes to the operating system kernel and userspace packages. In describing SELinux, the Mr. Faden makes this statement:
“SELinux, on the other hand, is an optional module that is included with the RHEL5 OS but is not part of most distributions, such as SUSE Linux.”
While true that SELinux is optional, it is part of the official 2.6 kernel and is supported as part of the default distribution of many applications. It is an optional feature of Linux, not a feature that is developed separately that must be added after the fact. It is also included in several distributions, notably Fedora, Red Hat Enterprise Linux, Debian and Gentoo. That might not be most distributions, but it is a good percentage of the major distributions.
Mr. Faden also makes this curious statement:
“Since SELinux must interface with the Linux kernel through the LSM interface, the aspects of access control that it can enforce are limited to those defined by the LSM interface rather than to all the requirements specified by the LSPP.”
The suggestion that there are LSM hooks that are missing that are required to meet “the requirements specified by the LSPP” is not correct. The SELinux developers have proposed the full set of hooks necessary for the LSPP and general security (and continue to propose new hooks when needed).
Policy Flexibility
Mr. Faden suggests several times that the flexibility SELinux policies is somehow harmful and supports this assertion by stating that:
“The rules for MLS data flow and for relabeling are precisely specified by the LSPP, so by definition, MLS is not a flexible policy.
The RHEL5 MLS policy configuration is an extension of the Linux type enforcement policy. As such, the policy is defined by using policy language primitives that are similar to those used to define types. In RHEL5 LSPP, it is necessary to explicitly declare the MLS rules for all object classes, since no inherent MLS policy is implemented in the kernel. Thus, for every application, file, user, network, or process that a customer wants to access, essentially anything on the system, a specific line in a security policy configuration file must be written. This can lead to extremely long and fragile security policy files that must be changed each time a new capability is exercised for an application that was tested during policy development. Each change in a security policy file requires a reload of the security policy in the kernel, which often requires a reboot of the RHEL5 system.”
There are several misconceptions in this quote.
- The rules for MLS data flow are specified by LSPP, but not how an operating system maps operations to data flows (e.g., reading, writing, and deleting files must be mapped to the MLS notions of read and write). The SELinux policy specifies that mapping, which is necessarily system dependent.
- The above language seems to imply that the user (or admin) will need to modify the MLS policy. This is not true. The flexibility is there for vendors or others making deep modifications to the operating system to make changes to how MLS maps to the operating system, but a normal admin will never make these changes. The flexibility is a good thing, as the alternative is hard-coding this information into the kernel. It in no way complicates the administration of the system.
- “Thus, for every application, file, user, network, or process that a customer wants to access, essentially anything on the system, a specific line in a security policy configuration file must be written.” This is not correct and I’m not certain where the misconception could have come from. Perhaps he is thinking about the type enforcement policy, but even there this is not true. Yes you can write rules about specific applications in type enforcement, but it is not nearly as long, complex, or fragile as this implies.
- I don’t know where this idea about policy changes requiring reboots came from, but it is not correct. I make policy changes all of the time without rebooting. Occasionally I have to restart a process, but only if I change its label. And very rarely I will reboot, but this is usually only when changing from one policy type to another (e.g., from targeted to strict). Even this is not strictly required, it is often just easier.
- “As of today, the SELinux MLS policy files are tens of thousands of lines”. This is not true – the full SELinux policy might be this long, the MLS portion is only about 200 lines long (without comments). The MLS policy also not new as the article implies in another section. This policy has been available and in development for several years.
SELinux Simplicity
Mr. Faden makes several comments about the complexity of SELinux, including:
“Simpler methods than type enforcement are available to confine applications, such as those provided by the Novell AppArmor for Linux product or by the CA eTrust for UNIX software for the Solaris 10 OS. Several OpenSolaris projects [7] are actively pursuing opportunities and solutions to provide additional per-process controls for file and network access.”
This is a long-running debate that I just want to touch on. Complexity can mean many things, including the kernel enforcement mechanism, application support, and policy. In SELinux, the kernel mechanism and application support are admirably simple. The policy is, I would argue, as simple as it can be.
SELinux attempts to be as simple as possible to address the fundamental issues in computer security, but no simpler. That means it must be capable of enforcing access control over all application operations involving the kernel (and in some cases, operations provided by applications like X or Dbus). In other words, the remaining complexity is fundamental to the problem not an artifact of the implementation. SELinux is exposing complexity that is inherent in an operating system as large and capable as Linux that supports large, and complicated applications. I believe that better high-level tools are the solution to dealing with this complexity, not crippling the underlying mechanism.
Other systems make simplifying assumptions at the expense of generality or completeness. The goal is to ease policy writing, but it is not clear to me that these simplifications in the model translate into a more understandable or maintainable policy. Ultimately the policy writer still has to deal with the details of the operating system and applications, which is where the real complexity resides.
Simplifications are attractive, but there is always a trade-off.
File Systems
Mr. Faden asserts that:
“RHEL5 LSPP requires customized versions of Linux file systems to associate security contexts with files and requires that the security context be specified for mounted file systems. The file systems are customized by extending the inode to include label information. Because of this strict requirement, new file systems and existing backup software must be specifically modified to support labels in order to work with RHEL5 LSPP.”
SELinux uses the generic Linux extended attributes, so file systems are not modified for SELinux. As extended attributes are used for many purposes in addition to SELinux (e.g., ACLs, beagle, etc.), there are many reasons to add support for extended attributes into backup and other file system utilities in addition to supporting SELinux. The native utilities used on Red Hat Enterprise Linux 5 (like tar and dump) support saving extended attributes.
Additionally, SELinux controls access to file systems that don’t support extended attributes through context mounts. So legacy filesystems (like FAT and NFS) can be used on SELinux systems. This support is very similar to how Trusted Extensions controls access to file systems in that every file and directory is treated as having the same label.
Performance
On the topic of performance, Mr. Faden says:
“Lmbench microbenchmark comparisons show that the overhead for SELinux file I/O operations is quite high (see “SELinux and grsecurity: A Case Study Comparing Linux Security Kernel Enhancements” [8]). Trusted Extensions adds no overhead to file I/O operations because it uses the unmodified, standard Solaris system calls, such as open(2), close(2), read(2), write(2), and stat(2).”
SELinux does impose some overhead on system calls and that overhead will be highlighted in a benchmark suite specifically designed to measure it, like Lmbench. Note that a performance hit in Lmbench means that the relative performance is lower, not that the overhead is high. Additionally, microbenchmarks, which are often aimed at system implementors, are not always a good indicator of overall performances. More general benchmarks designed to measure overall system performance consistently show that SELinux performance overhead is quite small.
The quote above strongly implies that Solaris with Trusted Extensions is better performing that SELinux without making that statement directly. However, it is important to note that comparing SELinux to Linux and Trusted Extensions to Solaris says nothing about the performance of SELinux compared to Trusted Extensions.
Finally, stating that Trusted Extensions adds no overhead to file I/O operations conveniently ignores the fact that it requires you to create a zone per-level. Since each zone is an OS instance, this overhead has to be quite high, particularly if you have several hundred or thousands of zones (which would not be an unusual number of active levels on a multi-level system according to users I have spoken with).
[Update: Both Mr. Faden and Mr. Comay suggest in the comments that I am implying that zones are heavier weight than they actually are. It is not clear to me what the performance impact of the zones really are (it may be quite low for the features that it has), but my point is that they do incur some performance impact and that they are heavier-weight than a plain process. I've been favorably impressed by what I've seen of zones, but nothing I've seen suggests that running hundreds or thousands of zones on a single system is a typical scenario. I'll be curious to see how they work in that type of configuration. ]
Conclusion
Ultimately I’m glad to see another approach to adding security to an operating system even if I do question the long term viability. I hope that SELinux and Solaris Trusted Extensions can cooperate where possible in upstream projects (like Xace), leading to better overall operating system security.
Updated 3/29/07: Corrected the name of Trusted Computer Solutions and made some minor edits for style based on feedback.
Updated 3/30/07: Responded to comments from Mr. Faden and Mr. Comay in body of the entry.
Hi Karl,
Good read but…
Your statement “bolt on security extensions for multi-level security” doen’t jive with “trusted variants included complex modifications to the very lowest levels of the operating systems”. It’s kinda glaring. Just thought I’d pass that along.
One clarification on the performance implications of Trusted Extensions using zones. Zones themselves do not implement a new layer and Glenn’s statement about file I/O performance is correct. In general, applications run inside a zone will see no performance degradation when compared to running on the system itself (or the global zone.) Much of this is due to the privilege model in Solaris 10 where the same checks for privilege operations used in the global zone are used by the non-global zones on the system.
Certainly creating a large number of zones can cause a general performance impact if there are applications running in those zones. The default set of applications is fairly small but if this is a concern, applying the OS resource management facilities prevents one zone from impacting the performance of the rest of the system or other zones on the system.
Thank-you for reading my article and posting this thoughtful response. I respect your opinion, but I want to ensure that facts are presently accurately.
There are several methods for upgrading of data between labeled zones. In addition to multilevel networking, we provide a doors-based service for reclassifying data which has a simple and a drag and drop GUI. We also support one-way named pipes for upgrading data.
Zones are more light-weight than implied in the article. For example, multilevel services such as NFS, printing, name service cache, login manager, session manager, and the window manager are shared by all zones. And zones can be created in seconds using ZFS cloning technology.
Your description of NFS as a legacy file system is surprising. In Trusted Extensions, remote file systems are labeled just like local file systems. A Trusted Extensions NFS server can export file systems at various labels, and enforces MAC appropriately, granting read or read-write based on client labels. The context mount feature of SELinux is a weak link because it depends on the end-user to identify the label of the remote NFS server.
When I referred to the size of the MLS policy I meant that it was a superset of the strict policy. You are correct that Trusted Extensions is a better way to create a trusted OS, but not about it being limited to government usage. It is appropriate in environments where inside and outside networks need to be safely accessed concurrently on a single desktop.
Thanks for the comments – I updated the article a bit in response.
In terms of NFS – calling it a legacy file system reflects my dislike of it and hope that it will eventually go away. NFS has rightfully earned a reputation for poor security and I think that NFS v4 is too little too late. Regardless, I’m not certain what “weak link” means. Mounting the shares will be done by an admin and, as NFS security is enforced by both the client and server, using context mounts does not pose a security risk. It does impose some administrative work, but otherwise I think that using context mounts in this way is a reasonable and secure way to serve NFS shares.
In terms of Trusted Extensions being useful for non-government customers: that argument has been made for years about trusted operating systems and there has been virtually no adoption outside of government customers. The problem, in my view, is that the separation offered by MLS systems is more severe than most commercial customers can tolerate and there is no secure way to relax those restrictions. That is where type enforcement really shines. It is possible to clearly and securely specify how information can flow to allow limited flow of data in certain circumstances through specific, trusted processes. The inflexibility of MLS doesn’t offer this: process are either confined by the policy or are trusted to circumvent it in coarse-grained ways. Relaxing the policy to make it useful essentially destroys any security benefit.
You wrote “Since each zone is an OS instance, this overhead has to be quite high” – Zone is NOT an OS instance actually.
NFS – you can configure secure NFSv3 on Solaris for years. It’s just Linux being behind.
NFS has had strong security for a long time (since long before NFSv4), both as a protocol and in several implementations (including Solaris). Sun first shipped Solaris support for using Kerberos V for NFS security around seven years ago! And it works. Support for better ciphers (particularly AES) in Kerberos and, therefore, NFS, shipped with S10.
Zones adds almost no overhead on file access code paths. The same applies to most access control paths in the kernel. That’s because zoned access checks begin by comapring the zone ID (and integer value) of the subject and the object, and that’s light weight indeed! Zones do have some overhead in the form of additional processes running in the background (e.g., SMF), but those processes are few, rarely run and don’t get in the way of the access controls that you were talking about. Whereas SELinux file access controls must be slower than unlabelled ones because of the need to read per-file labelling information from extended attributes; the filesystem could play clever inheritance games to reduce this overhead in most cases, but from the above it seems that your filesystems don’t (right?).
Wow – who knew there were so many NFS fans?
I don’t quite know what keeps going wrong about this zones performance discussion. Let me try one more time. All I am saying is that zones have a) some impact on system calls and b) some additional overhead in terms of resources (memory, mounted file systems, processes, etc.). In terms of a, I’m not saying that it is a high overhead, just that it isn’t free. File access might be the same whether you are in a zone or not, but that doesn’t mean that there isn’t some overhead, however small. For b, again, I’m not saying that the overhead is high, I’m saying that it is there. When you start talking about instantiating hundreds or thousands of zones that resource usage starts to add up.
This is all in reaction to Mr. Faden’s article which, to me, seemed to try to imply that there was a performance impact for SELinux but not for Trusted Extensions. My only point is that is not true, not that the overhead of zones is particularly high.
For SELinux there is some overhead, but it is certainly not reading the labeling per-access as Nico says above. The label is read once and cached from that point forward. If you look closely at the implementation of SELinux you will see that the overhead of most accesses is very low because of the access vector cache. There are a few code paths, like the old network controls, that didn’t use that cache. Those paths had higher overhead, but most have been removed by this point.
I know that Solaris vs. Linux performance is one of those things that people love to debate. I don’t really want to get in the middle of that. I just want people to understand that both approaches have some overhead and encourage potential users to benchmark their workloads to see which platform meets their performance needs.
Zone are almost as light-weight as SELinux domains, but they completely solve the polyinstantion problem which has plagued legacy MLS systems. For example, two instances of Apache at different labels can each bind to port 80 on the same IP address, but only receive requests at their own label. Can SELinux do this?
With respect to NFS, I don’t understand why it is necessary for the administrator to specify the label of a remote NFS filesystem. It seems error-prone and redundant. In TX the label is determined by the kernel, based on the trusted networking policy.
I certainly agree that SELinux can solve some problems better than Trusted Extensions, just as the reverse is true.
But the SELinux implementation is currently too primitive. It lacks support for centralizing the policy in a nameservice, has an incomplete labeled networking implementation, has no labeled desktop, and has insufficient fine-grained policy for user authorizations.
Of course Solaris isn’t perfect. We are working on extending our RBAC framework to provide more fine-grained access controls, and ease of use can surely be improved.
But the point of my article, and yours, is that both Trusted Extensions and SELinux offer new solutions for MAC. Obviously I chose to focus on MLS because that is the essential feature of Trusted Extensions. In the case of SELinux, it seems to me that MLS is only offered to satisfy evaluation criteria, rather than being a useful configuration. I haven’t heard anyone in the SELinux community state otherwise.
Some responses to what Mr. Faden says in the comment above.
* “Zone are almost as light-weight as SELinux domains” – this doesn’t make sense to me. SELinux domains are just security properties on processes that get checked by the SELinux security server. Zones may be light-weight, but they can’t be as light-weight as a process.
* SELinux consciously decided against polyinstantiation of most resources. The same effect can be achieved through different means with significantly less complexity.
* SELinux could use labeled networking to determine the label of the remote filesystem, it is just no one has done that integration. The advantage to specifying the label, of course, is that legacy NFS servers work fine.
* “I certainly agree that SELinux can solve some problems better than Trusted Extensions, just as the reverse is true.” Here is where I disagree – SELinux solves a superset of the problems that Trusted Extensions solves.
* “It [SELinux] lacks support for centralizing the policy in a nameservice”. This is in progress in the open source community, but is supported now through a 3rd party product (http://tresys.com/products/brickwall.html).
* “[SELinux] has an incomplete labeled networking implementation”. No idea what you mean. Cipso, IPSec labeled Networking, and secmark all work and are fully implemented.
* “[SELinux] has no labeled desktop”. This was addressed in the article. This is in-progress in open source and available now through a 3rd party (http://tcs-sec.com/).
* “[SELinux] has insufficient fine-grained policy for user authorizations”. No idea what you could possibly mean here. SELinux has flexible and comprehensive control over users and solves problems that are not solvable using privileges.
* “In the case of SELinux, it seems to me that MLS is only offered to satisfy evaluation criteria, rather than being a useful configuration. I haven’t heard anyone in the SELinux community state otherwise.” Have you asked anyone in the community? The MLS extensions are offered to solve the needs of those people that really need MLS (i.e., government customers with a large number of levels). It is a useful configuration for them and considerable effort was invested in making it useful. Evaluation just makes it possible for those customers to use RHEL.
Please stop spreading FUD about SELinux, particularly on my blog.
Good article, Karl, and I enjoyed the dialogue as well. On a slightly different note, your primary distinction in this entry is that a Trusted OS is a separate version of a mainstream OS, thus your statement that RHEL 5 is not a trusted OS and your desire that it does not become a Trusted OS.
While it may be a semantic argument, I think it is an important one: an application is “trusted” based on what it is doing – one could argue that all operating systems are “trusted” because they are performing security relevant functions. From a security perspective, what is important is how trustworthy an application (or OS) is, particularly one performing a security relevant function. This is why assurance is so important in some customer spaces – it is important to know that the code actually does what it claims to do, and nothing else. Of course 100% assurance is not completely achievable for useful software, which is why there are levels of assurance reflecting the degree of confidence.
What many people think when they hear “Trusted OS” is “trustworthy OS”, and the two are not always synonymous (though a good marketing approach!).
Despite the semantics, what my customers need is a trustworthy OS offering the appropriate mechanisms to meet their security needs. I believe RHEL fits the bill. I regularly fight the semantic battle of clarifying what makes a “Trusted OS”, and a separate development and maintenance branch is a significant disadvantage, as you noted. Unfortunately, when my customers hear that RHEL 5 is not a Trusted OS, I have a lot of explaining to do!
thanks to all who contributed; very interesting and informative (if wrangling with the political) article and comments.
i won’t drag more products (and politics) into the discussion but can say that we have gained good ground on trusted extensions based, not only 1-to-1 mapping of zones to labels but more significantly, that a number of customers have hit significant performance and management bottlenecks once zone enumeration starts to exceed 200 and in one discussion a project manager indicated that it “tops out” implying to me that this is not a linear problem.
i accept that this is loose and un-qualified (particularly in respect of hardware/configuration) and can only apologise for that (i am severely restricted in what i may say) but perhaps it will draw people out on more specific numbers.