Support for Building Elements in the IFC 2x3 Implementations based on 6th Certification Workshop Results in May 2007
Arto Kiviniemi, Research Professor of ICT for BuiltEnvironment, VTT
Introduction
The information in this document is based on the 6thIFC 2x3 Certification Workshop held in May 2007 in Espoo, Finland. Thomas Liebich sent thedetailed draft results to the authoron September 18th and 19th, 2007in two files:
- IFC2x3CertificationStep1_02-BuildingElements_ws6_draft.xls
- IFC2x3CertificationStep1_01-BasicTests_ws6_draft.xls
This document contains simplified conclusions from these documents related to the building elements and spaces. Annotations, which are basically 2D drafting elements, are not in the scope of this study.
Although all presented applications have IFC 2x3 certification after the workshop, it was not the final certification. This means that some of the highlighted problems in the software applicationswill probably be only temporary; all certified applications have still some remaining issues, which, based on the information on the ISG web site, “have to be resolved in the near future”.In addition, some conclusions may base on misinterpretation of the results because the information of the certification did not include all details. However,more information is not available at the moment. Thus, the document will be distributed for comments and any mistakes will be corrected as soon as possible, after more information will be available.
In addition it is notable that certification is a complex process; balancing between what“is acceptable for end-users and economically achievable for vendors. If the certification can only be passed, when no issues are left, there will never be certified applications. As there is no bug-free software available in the world, there will never be bug-free IFC-interfaces. …Due to lack of interest in the industry, it is a huge problem to find enough of pilot-users for the IFC-interfaces, especially for cross-testing between the various applications.”[Steinmann, 2007]. Thus, passing certification must include the question what an acceptable balance is. In the end this is something that the end-users will decide by adopting or discarding the IFC data exchange with certified applications.
Certification process
The current certification process is based on the Extended Coordination View for IFC 2x3 and software products mentioned in the certificationdocumentsare documented in Table 1.
Application / Release / Vendor / 1st step / 2nd step / CommentActive3D / v 4.0 / Archimen / 06/2006 / 03/2007 / Import only
Allplan / 2006.2, 2008 / Nemetschek / 06/2006 / 03/2007
ArchiCAD / 11 / Graphisoft / Nemetschek / 06/2006 / 03/2007
Bentley Architecture / 8.9.3 / Bentley / 06/2006 / 03/2007
Revit Architecture / 6.4 / Autodesk / 06/2006 / 03/2007
Solibri Model Checker / Solibri / 06/2006 / 03/2007 / Import only, no 2D
TEKLA Structures / 13 / TEKLA / 06/2006 / 03/2007 / No 2D
AutoCAD Architecture / 2008 / Autodesk / 11/2006 / 03/2007
House Partner / 6.4 / DDS / 03/2007 / 03/2007
MagiCAD / Progman / 03/2007 / 05/2007
Facility Online / Vizelia / 05/2007
IFC3DX / NorConsult / 05/2007 / Import only
SCIA-ESAPT / SCIA / Nemetschek / 05/2007
VectorWorks / Nemetschek NA / 05/2007
Table 1: Software products in the IFC 2x3 certification ( Source: ISG web site at )
The applications marked by the grey background in the table are not handled in this paper since most of them were in the early stages of certification and therefore not yet properly tested. A strange issue in the documentation was that there were no results of Progman’s MagiCAD although the above table and also recorded results on the ISG web site indicate that MagiCADhas already passed the 2nd step certification. Because of the lack of documented results MagiCAD could not be included in this paper at this stage.
Official certification results based on the ISG web site
5thWorkshop: March 12th – 14th, 2007 in Waltham
1st round applications: Allplan, ArchiCAD, Bentley Architecture, Revit Architecture, TEKLA Structures, Active3D and Solibri Model Checkerpassed the 2nd-Step-Certification with the condition, that remaining issues have to be resolved in the near future.
2nd round applications: AutoCAD Architectureand House Partner successfully passed the 2nd-Step Certification based on IFC2x3 and the Extended Coordination View under the condition, that remaining issues will be solved in the near future.
3rdWorkshop: May 21st – 23rd, 2007 in Espoo
MagiCADsuccessfully passed the 2nd-Step Certification based on IFC2x3 and the Extended Coordination View.
Comments of the author
Based on the information on the official ISG (Implementation Support Group) web site, 10 applications have passed the IFC 2x3 Extended Coordination View Certification. However, all of them conditionally requiring that the “remaining issues will be solved in the near future”. Only 9 applications are compared in this paper since the information of MagiCAD’s results was totally missing from the certificationdocuments.
The certification concentrates mainly to IFC import. There is no systematic testing of IFC export. The only part testing export is the creation of the test files, but since the test cases are different and there is no clear documentation how their structure and content have been defined and by whom, it is possible that the cases are selected based on the known capabilities of each exporting software and thus not testing the potential problem areas.
Another issue is that all test cases are very small independent files representing some individual building element or a small group of such elements. None of the tests handles a whole building. Unfortunately, several tests using earlier IFC versions and larger files indicate that certification using this method does not guarantee the functionalities with real buildings. However, regarding IFC 2x3 implementations this is just an assumption;the potential problems cannot be judged without a thorough testing.
The available certification documentation does not include information of property sets. Thus, the question how the support for property sets in different objects is implemented is at the moment open.
The test cases exported by different applications in each category are documented in Table 2. It is noteworthy that some applications are certified for import only and thus not included in this group. Some test cases include information of potential export errors, but this part is not discussed in this paper since the export cases seem to be rather arbitrary.
Archi-CAD / Allplan / AutoCAD Arch. / Bentley Arch. / Revit Arch. / SCIA / Tekla / Other / In totalSpaces / 1 / 2 / 2 / 3 / 0 / 0 / 0 / 0 / 8
Walls / 22 / 18 / 13 / 23 / 19 / 5 / 3 / 1 / 104
Beams / 3 / 6 / 4 / 6 / 4 / 4 / 6 / 3 / 36
Columns / 5 / 3 / 3 / 5 / 4 / 4 / 7 / 1 / 32
Slabs / 5 / 3 / 4 / 4 / 7 / 2 / 4 / 0 / 29
Doors / 2 / 3 / 6 / 2 / 6 / 0 / 0 / 2 / 21
Windows / 2 / 5 / 7 / 2 / 3 / 0 / 0 / 0 / 19
Stairs / 2 / 1 / 1 / 2 / 2 / 0 / 0 / 0 / 8
Ramps / 2 / 0 / 1 / 3 / 2 / 0 / 0 / 0 / 8
Railings / 1 / 0 / 1 / 2 / 1 / 0 / 1 / 0 / 6
Roofs / 3 / 2 / 3 / 3 / 2 / 0 / 0 / 0 / 13
Curtain walls / 2 / 1 / 2 / 1 / 1 / 0 / 0 / 0 / 7
Members / 1 / 0 / 2 / 2 / 0 / 0 / 3 / 1 / 9
Plates / 1 / 0 / 0 / 0 / 0 / 0 / 3 / 0 / 4
Piles / 1 / 0 / 0 / 1 / 0 / 0 / 1 / 0 / 3
Footings / 1 / 1 / 0 / 2 / 0 / 0 / 1 / 0 / 5
In total / 54 / 45 / 49 / 61 / 51 / 15 / 29 / 8 / 312
Percentage / 17 % / 14 % / 16 % / 20 % / 16 % / 5 % / 9 % / 3 % / 100 %
Table 2: Test case distribution by the exporting software application
Building Elements
Overall results
The total number of used test cases in the certification was 312 divided in 16 groups: Spaces (8), Walls (104), Beams (36), Columns (32), Slabs (29), Doors (21), Windows (19), Stairs (8), Ramps (8), Railings (6), Roofs (13), Curtain Walls (7), Members (9), Plates (4), Piles (3) and Footings (5).
The certification results must be interpreted since all details were not recorded in the available documentation. This report regards the results accepted if the result were marked “OK”, “Geometry and connections OK”, “Geometry and materials OK” or “Geometry OK” without any major comments i.e. at least model geometry can be exchanged, but it is not possible to say if some other type of data may be missing. Unfortunately the certification documentation does not include specific information what may be missing in the cases, where the marking is something else than simple “OK”. The passing criteria are not documented, and since testing is done by several people, the criteria can vary between the software applications.
One important aspect is also that some of the test cases represent very common geometry in buildings and some are rather unusual, i.e. not often used in building design. Thus, the statistical approach does not necessarily correlate with the ability of different applications to exchange IFC data in real projects. However, since there is no weighting of the importance of different test cases and the participating implementers have approved all test cases into the official certification process, the statistical approach and some additional view points, such as deviation of results, were selected as the methods to evaluate the results.
In addition, the software documented in the certification files are in different phases of the certification, some have been tested with all or almost all test cases, some have very few or no documented results in the files. This paper concentrates only to those applications which have officially passed the 2x3 certification according the ISG web site and have over 75% average passing rate. Others are considered to be in the early stage of certification and thus the results are not yet reliable and may give a false image of the software.
Results by element type
Figure 1: Results by the element type
In Figure 1legend “All OK” means the percentage of test cases which all 9 applications have passed in the certification, “8/9 OK” means the percentage of test cases which at least 8 applications have passed in the certification. This is documented to see the impact which bad results in one application may have caused; a typical example of this are spaces where “All OK” is 13%, but “8/9 OK” is 63%. “Average” means the average number of test cases passed in each category.
The results vary significantly. In total 51% of the test cases passed in all applications and 69% in 8 of the 9 applications. Compared to this, the average passing rate of different test cases is relatively good, 87%. In each element categoryat least some test cases passed in all applications. When looking average results (OK average) the situation looks also relatively good; the results vary between 68% (Windows) and 100% (Piles). However, the results indicate one major problem: the deviation of test cases passing or failing is very high. If only about 50%of the test cases in all element categories pass in all applications, it means that the end-users cannot rely on the IFC exchange of the certified applications.
Spaces / Active3D / Allplan / ArchiCAD / AutoCAD Arch. / Bentley Arch. / House Partner / Revit Arch. / Solibri / TeklaTest 1 / + / + / + / - / + / + / + / + / -
Test 2 / + / + / + / + / + / - / + / + / +
Test 3 / + / + / + / + / + / - / + / + / +
Test 4 / + / + / + / + / + / - / + / + / +
Test 5 / + / + / + / + / + / + / + / + / +
Test 6 / + / + / - / + / - / - / - / + / +
Test 7 / + / - / - / - / - / - / + / + / +
Test 8 / + / + / + / + / + / - / + / + / +
Table 3: Passing rate of space test cases in different applications
Unfortunately, the results in some element categories are significantly worse because only very few test cases in these categories pass in all applications: spaces (13%), doors (14%) and windows (16%). The piles are the only element category where all test cases pass in all applications. Tables 3-5 illustrate the deviation in these object classes. It is very difficult for end-users to guess in which cases the space, door and window data can be exchanged without some missing or misrepresented information. This makes the data exchange in real projects extremely demanding, if not impossible.
Doors / Active3D / Allplan / ArchiCAD / AutoCAD Arch. / Bentley Arch. / House Partner / Revit Arch. / Solibri / TeklaTest 1 / + / + / + / + / + / + / + / + / +
Test 2 / + / - / + / + / + / + / + / + / +
Test 3 / + / + / + / + / - / - / + / + / +
Test 4 / + / + / - / - / - / + / + / + / +
Test 5 / + / + / + / - / - / + / + / + / +
Test 6 / + / + / + / + / + / + / + / + / +
Test 7 / + / + / + / + / - / - / + / + / +
Test 8 / + / + / + / + / - / - / + / + / +
Test 9 / + / + / + / + / - / + / + / + / +
Test 10 / + / + / + / - / + / - / - / + / +
Test 11 / + / + / - / - / - / - / - / + / -
Test 12 / + / + / + / - / - / + / + / + / +
Test 13 / + / + / + / - / - / + / - / + / +
Test 14 / + / + / + / + / + / + / + / + / +
Test 15 / + / + / - / - / + / + / + / + / +
Test 16 / + / + / - / - / - / + / - / + / +
Test 17 / + / + / + / - / + / - / + / + / +
Test 18 / + / + / + / - / + / - / - / + / +
Test 19 / + / + / + / - / - / - / - / + / +
Test 20 / - / + / - / - / - / - / + / + / +
Test 21 / - / - / - / + / + / - / - / + / +
Table 4: Passing rate of door test cases in different applications
Windows / Active3D / Allplan / ArchiCAD / AutoCAD Arch. / Bentley Arch. / House Partner / Revit Arch. / Solibri / TeklaTest 1 / - / - / - / - / - / - / + / + / +
Test 2 / - / + / - / + / + / - / + / + / -
Test 3 / + / + / + / - / - / + / - / + / +
Test 4 / + / + / + / - / - / + / - / + / +
Test 5 / + / + / + / + / + / + / + / + / +
Test 6 / + / + / + / + / + / + / + / + / +
Test 7 / + / + / + / + / + / + / + / + / +
Test 8 / + / + / + / - / - / + / - / + / +
Test 9 / + / + / + / - / - / + / - / + / +
Test 10 / + / + / - / + / - / - / + / - / +
Test 11 / + / + / + / + / - / + / + / + / +
Test 12 / - / + / + / + / - / - / + / + / +
Test 13 / + / + / - / + / + / + / - / + / +
Test 14 / + / + / + / - / - / + / - / + / +
Test 15 / - / + / - / + / - / - / + / + / +
Test 16 / - / - / - / + / - / - / + / + / +
Test 17 / + / - / - / - / - / - / - / + / +
Test 18 / + / + / + / + / + / - / + / + / +
Test 19 / + / + / + / - / - / - / - / + / +
Table 5: Passing rate of windows test cases in different applications
Results by software
When looking how well different applications have passed the test in average (Figure 3), four of the applications are clearly above the others – Solibri (96%), Active 3D (94%) Tekla (93%), and Allplan (92%). However, none of the results are really bad compared to the results of earlier certification workshops; the lowest passing rates are now 79% (AutoCAD Architecture and Bentley Architecture).This indicates that the certification process consisting of several steps and workshops is improving the quality of IFC interfaces. In House Partner results the beams have ignored in the results since they are not in the scope of the software.
Figure 3: Average results by software
Active-3D / Allplan / Archi-CAD / AutoCAD Arch. / Bentley Arch. / House Partner / Revit Arch. / Solibri / Tekla
Spaces / 100 % / 88 % / 75 % / 75 % / 75 % / 25 % / 88 % / 100 % / 88 %
Walls / 92 % / 89 % / 85 % / 85 % / 85 % / 93 % / 85 % / 95 % / 92 %
Beams / 97 % / 94 % / 100 % / 97 % / 94 % / 94 % / 97 % / 94 %
Columns / 94 % / 100 % / 100 % / 81 % / 97 % / 72 % / 94 % / 97 % / 88 %
Slabs / 100 % / 100 % / 79 % / 76 % / 76 % / 86 % / 86 % / 93 % / 90 %
Doors / 90 % / 90 % / 71 % / 43 % / 43 % / 52 % / 67 % / 100 % / 95 %
Windows / 74 % / 84 % / 63 % / 58 % / 32 % / 53 % / 58 % / 95 % / 95 %
Stairs / 100 % / 75 % / 63 % / 88 % / 75 % / 100 % / 88 % / 88 % / 100 %
Ramps / 100 % / 100 % / 88 % / 75 % / 88 % / 88 % / 75 % / 100 % / 100 %
Railings / 100 % / 83 % / 83 % / 83 % / 83 % / 83 % / 83 % / 100 % / 100 %
Roofs / 92 % / 92 % / 62 % / 62 % / 69 % / 69 % / 62 % / 92 % / 92 %
Curtain walls / 100 % / 100 % / 71 % / 57 % / 71 % / 100 % / 71 % / 100 % / 100 %
Members / 100 % / 78 % / 78 % / 78 % / 67 % / 67 % / 78 % / 78 % / 100 %
Plates / 100 % / 100 % / 100 % / 100 % / 100 % / 100 % / 100 % / 100 % / 75 %
Piles / 100 % / 100 % / 100 % / 100 % / 100 % / 100 % / 100 % / 100 % / 100 %
Footings / 100 % / 100 % / 80 % / 100 % / 100 % / 100 % / 100 % / 100 % / 100 %
In total / 94 % / 92 % / 83 % / 79 % / 79 % / 80 % / 83 % / 96 % / 93 %
Table 6: Results by software and building element
The detailed results by each application and each building element category are documented in Table 6. Active 3D passes in 10 categories with 100% rate, Solibri in 8 categories, Tekla and Allplan in 7 categories. In addition, none of these applications has low passing rate in any category; Active 3D, Solibri and Tekla have each one result below 80%, Allplan has two. In the other end of spectrum are AutoCAD Architecture, Bentley Architecture and House Partner which all have several categories below 60% passing rate. The lowest scores are Bentley Architectures 32% passing rate in Windows category andHouse Partner’s 25% in Space category.
As a conclusion based on the available information it seems that the best four applications have relatively small deviation in the results. However, as stated earlier these results using small test cases do not necessarily correlate with ability to exchange IFC data in real projects.
Spaces
8 test cases, all relatively simple and only one including the bounding wall elements.
Only 1 test case (13%) passed in all applications, 5 (63%) in 8 of the 9 applications, 79% passing average. House Partner has very low passing rate (2 cases, 25%) and it is the main reason for the bad result regarding all applications. When its results are ignored, the support for spaces is relatively good.
All tested applications support basic spaces including geometry and both short and long name. The support covers also curved walls and simple voids, such as columns and in most applications also at least simple clipping. Revit supports only 2D representation of spaces since it does not have 3D space geometry in its internal structure. However, this has been evaluated here as passing the test since the spatial information seems to be imported correctly to Revit’s own internal 2D structure although this is not very clearly recorded in the certification documentation.
Walls
104 test cases, including different joints and openings, some relatively complex.
58 test cases (56%) passed in all applications, 80 (77%) in 8 of the 9 applications, 89% passing average.
All applications pass the majority of the test cases. However, in several applications complex openings, recesses and clipping planes can cause missing information or other problems. In some complex cases the wall layers can be flipped in some applications. A notable detail is that Tekla does not support wall layers, similarily as Revit does not support 3D spaces.
Based on the certification documents it is not clear if the wall type information is part of the IFC 2x3 Extended Coordination View, at least it is not documented in the test cases. The problem is related to the general missing documentation of the property sets, mentioned in section “Certification process”. This can mean that the type information cannot be exchanged without additional agreements in which property set to store it. In general, this is in my personal opinion a major problem since type information could simplify data exchange by replacing,for example, the need to use material layers.
Beams
36 test cases, including basic geometry, parametric steel profiles, brep shapes, openings and clipping.
28 test cases (78%) passed in all applications supporting beams, 96% passing average. House Partner does not include beams so it is ignored in this element category.
In general, beams seem to be one of the element categories which are well supported in the certified applications.
Columns
32 test cases, including basic geometry, parametric steel profiles, brep shapes, openings and clipping.
15 test cases (47%) passed in all applications, 25 (78%) in 8 of the 9 applications, 91% passing average.
Openings in columns seem to be a problem in some applications; otherwise columns seem to be one of the element categories which are well supported in the certified applications.
Slabs
29 test cases, including basic geometry, material layers, openings, recesses and clipping.
15 test cases (52%) passed in all applications, 18 (62%) in 8 of the 9 applications, 87% passing average.
Openings in slabs seem to be a problem in some applications. Allplan and Bentley do not support layers in slabs; Bentley creates individual slabs from each material layer.
Doors
21 test cases, including parametric opening directions, explicit geometry and different operation types.
Only 3 test cases (14%) passed in all applications,5 (24%) in 8 of the 9 applications, 72% passing average.
Miscellaneous errors, for example, wrong positions and geometry, missing faces and even doors. Deviation is very high and results in data exchange can be very unpredictable. Doors seem to be a very problematic object type in the IFC exchange, possibly because of very different internal representations.
Windows
19 test cases, including parametric shape representation, explicit geometry and different operation types.
Only 3 test cases (16%) passed in all applications, 5 (26%) in 8 of the 9 applications, 68% passing average.
Miscellaneous errors, for example, wrong positions and geometry and missing geometry andoperation types. Deviation is very high and results in data exchange can be very unpredictable. As doors, also windows seem to be a very problematic object type in the IFC exchange, possibly because of very different internal representations. Active3D and Tekla do not support parametric windows. However, Tekla announced that they will be supported later.