IDS.py toolkit as a result of Google Summer of Code 2021

 

This year ifcopenshell.org, and the umbrella mentoring organisation brlcad.org took part in the Google Summer of Code project (summerofcode.withgoogle.com/), where students contribute to open-source under the wings of experienced mentors. Out of many proposals contributing to IfcOpenShell, two projects were selected. One of them is the development of the IDS toolkit by Artur Tomczak with support from Thomas Krijnen, that is a response to the issue: github.com/IfcOpenShell/IfcOpenShell/issues/1349.

IDS – Information Delivery Specifications – is a standard being development for defining BIM information requirements in an easy to present way in both machine and human-readable form. With IDS you can specify what data must be included in the BIM model and validate if it is delivered. IDS is related to the Level of Information Need that can be used in project documentation (OIR, PIR, AIR, EIR, BEP, TIDP, MIDP, MPDT etc.). Finally, it is the solution for predictable and reliable data exchange workflows. Read more about IDS at technical.buildingsmart.org/projects/information-delivery-specification-ids/ and technical: github.com/buildingSMART/IDS.

During the project, we were able to complete objectives according to the plan. The toolkit now fully supports IDS authoring, allowing to define all possible requirements and restrictions. It also supports the validation of the IFC models against such IDS files. As a result of the validation, one can print all the issues in the model or save them as BCF report and view them in the 3D viewer. To get started with the toolkit have a look at the documentation: ifcopenshell.github.io/docs/python/html/ifcopenshell-python.html.

As one of the first implementations of the standard, the project contributed to the improvement of the standard, which resulted in two schema updates. The newly created XSD standard is available at: http://standards.buildingsmart.org/IDS/ids_04.xsd. 

To help users create own IDS without installing the tool and for non-coding users, the web application has also been developed that works thanks to the Python toolkit. It is accessible at artomczak.pythonanywhere.com/create/. You can define your specifications with applicability filters and requirements that your BIM model must fulfil and download it as an XML file.

Project deliverables:

·        Main work is available at GSoC#45-IDS-checking branch: github.com/atomczak/IfcOpenShell/tree/GSoC%2345-IDS-checking/src/ifcopenshell-python/ifcopenshell.

o   The IDS toolkit is enclosed in the ids.py file (github.com/atomczak/IfcOpenShell/blob/GSoC%2345-IDS-checking/src/ifcopenshell-python/ifcopenshell/ids.py).

o   Test cases showing possible usage of the toolkit are in the file test_ids.py (github.com/atomczak/IfcOpenShell/blob/GSoC%2345-IDS-checking/src/ifcopenshell-python/ifcopenshell/test_ids.py).

o   The full list of commits: github.com/atomczak/IfcOpenShell/commits/GSoC%2345-IDS-checking.

·        Test cases are supplemented with a repository of sample IDS and IFC files available at: github.com/atomczak/Sample-BIM-Files.

·        The web page is hosted at: artomczak.pythonanywhere.com/.

·        The web interface source code is kept in another repository: github.com/atomczak/IDS-web-app. All the current limitations of the web app are listed in the repository’s issues.

If you have any questions or comments, please join the discussion at OSArch thread (https://community.osarch.org/discussion/647/).

 

A new year; a new IfcOpenBot

Compilation of large C++ projects takes patience so for a while now we have been provided automated builds to the community using a dedicated physical system we called IfcOpenBot. In reality this poor thing was a Mac Mini from 2014 in the least powerful configuration possible, sitting next to the meter box at the AECgeeks office. The reason for this setup is that compiling for macOS requires a macOS operating system and a running a macOS virtual machine on non-apple hardware is a violation of their terms of service. We filled the Mac Mini with VirtualBox virtual machines (including a cheap Win 8 Pro license that got a free upgrade to Win 10) and tied things together with a bunch of shell scripts (VBoxManage to start and stop the VMs) and SSH to connect into the hosts. We installed Cygwin into the windows box to allow SSH access. Then it was simply a matter of running the build scripts we have in the repository and collecting the output.

This approach worked well for several years, but began to face issues:

• The physical setup and MacOS host didn't really allow for full remote access, this meant we had to carry monitors around occasionally to connect and debug.
• The setup of VirtualBox incremental snapshots introduced a massive storage overhead. So more and more portable USB SSD drives were added.
• Build times were really slow, even with caching the dependencies and using ccache it took several days.
• In the end it wasn't really possible anymore to run the latest Win 10 guest anymore along the macOS host as they were fighting for the little 4GB of RAM.

New approach

The christmas holidays gave us some time to rework this solution.

We rent a hosted Mac Mini using one of the commercially available solutions. Monthly cost $60.
We have a digital ocean droplet doing the orchestration. Monthly cost $5.
Windows and Linux machines are spinned up using the AWS EC2 command line tools. Build on linux takes about 1 hour on t2.2xlarge so that's 30 dollar cents per build according to the current prices.
For windows we have configured the user-data scripts to install chocolatey and WinRM on boot and on the host we use ansible to control the build. For the other systems we just use plain SSH.

The new builds from this system have just arrived: please help test for issues. In addition to the changes discussed above, the linux distro is changed from Debian to Ubuntu 16.04 (so this changes glibc versions and all). Since EC2 does not host 32bit operating systems (anymore, it seems), we use cross-compiling now on 64bit Ubuntu. The visual studio version on Windows is now 2017 instead of 2015 (using the build tools package from chocolatey).

Support

Among others, this shows the complexity for open source projects to build for macOS. Initially it drove us to expensive hardware that we couldn't upgrade. In the current approach it takes up over 85% of the cost (assuming a weekly build on EC2).

If the IfcOpenBot builds are useful to you, please consider a small donation at https://opencollective.com/opensourcebim to cover these costs. OpenSourceBIM is a new foundation started by myself and Dion Moult that tries to organize and support the development of open source software for OpenBIM. We will provide more details later.

v0.6.0

Many people perhaps don't follow the development of IfcOpenShell actively and are happily using the master branch of the github repository. They might be surprised to know there is a lot of activity happening in the v0.6.0 and v0.7.0 branches. This post discusses the changes in the v0.6.0 branch. The following post will elaborate on some of the design decisions we are making in the v0.7.0 branch.

Schemas

One of the most significant improvements in the v0.6.0 branch is that multiple schemas (IFC2X3, IFC4, IFC4X1 and IFC4X2) are supported from within the same executable, module or plug-in. Previously, selecting the schema had been a compile-time option.

In IfcOpenShell and most other EXPRESS-based toolkits, the IFC schema is compiled into (a) the early-bound definitions: a class hierarchy with member functions and (b) a set of methods to operate on the schema definitions at runtime (late-bound access). C++ only allows very limited runtime reflection (but the development of C++ is very active, see for example P1240) so to complement the lack of introspection a set of methods exists to query for example all attribute names or the sub- and supertypes of an entity. In the master branch these methods are static, in the v0.6.0 branch these are the member functions of a schema class, that is a more complete reference mirroring the EXPRESS schema definition at runtime. See

IfcBaseEntity::declararation()

or

IfcParse::schema::declaration_by_name("IfcWall")->as_entity()->all_attribute_names()

Writing schema agnostic code

The code generated from the four schemas are completely orthogonal class hiercharies. For the C++ compiler there is no relationship between a Ifc2x3::IfcWall and a Ifc4::IfcWall. But IfcOpenShell offers three ways to write code that adapts to the schema of the file at runtime.

(a) preprocessor

This is the approach taken in the IfcGeom modules in v0.6.0. Essentially the same code base is compiled multiple times where the schema is available as a preprocessor constant. This means you can enable specific code paths with for example #ifdef directives. In this way the added entities in Ifc4 (IfcBSplineSurface, yay!) can be selectively compiled for example.

https://github.com/IfcOpenShell/IfcOpenShell/blob/e283b51dbcced6d8121c55fafd49c9ee1f954b74/src/ifcgeom/IfcGeomFaces.cpp#L1147

Smaller code blocks can be written as macros as well.

https://github.com/IfcOpenShell/IfcOpenShell/blob/e283b51dbcced6d8121c55fafd49c9ee1f954b74/src/ifcgeom_schema_agnostic/Kernel.cpp#L74

IfcOpenShell uses CMake to create multiple shared libraries from the same code, see https://github.com/IfcOpenShell/IfcOpenShell/blob/e283b51dbcced6d8121c55fafd49c9ee1f954b74/cmake/CMakeLists.txt#L557 for the foreach loop creating multiple libraries with different directives.

Benefits: fairly readible code, full autocompletion typically in an IDE when using the static library approach
Downsides: Some infrastructure required to compile the different libraries and select the correct implementation at runtime

(b) late-bound access

There are two modes of accessing schemas. In the early-bound approach function signatures and return types are known at compilation time. In the late-bound approach attribute names are referenced by strings and types are (basically) a tagged union of all data types used in the IFC schema, mapped to C++ types.

Ifc2x3::IfcWall* wall;
// Early-bound access;
{std::string global_id = wall->GlobalId();}
// Late-bound access.
{std::string global_id = *wall->get("GlobalId");}
// ERROR: By dereferencing the return type, it is cast into a string, which will cause an exception *at runtime* when the types do not match.
{int global_id = *wall->get("GlobalId");}

Benefits:
fairly readible code
no complicated setup of different libraries
Downsides:
no code completion
errors are only spotted at runtime, not compile-time
late-bound manipulation of inverse attributes is not well supported currently in IfcOpenShell
less means for the compiler to create highly optimized code

(c) templates

C++ has very extensive support for compile time generic arguments: templates. For this purpose the Ifc2x3 and Ifc4 definitions are no longer namespaces (in v0.5.0) but are now structs to allow for dependent names.

template <typename Schema>
void print_globalid(typename Schema::IfcWall* wall) {
    std::cout << wall->GlobalId();
}

Benefits:
no complicated setup of different libraries
no autocompletion typically, but errors caught at compile-time
Downsides:
fairly unreadable code due the necessity to sprinkle the code with additional template and typename keywords throughout.
error messages are harder to make sense up (due to two phase lookup rules for example)

All three approaches are used in the IfcOpenShell code-base.

Other improvements:

Multi-threading in collaboration with TNO, MAUC and Airsquire

Direct binary glTF v2.0 output (previously supported through Collada and Collada2Gltf) in collaboration with Schuco US.

Exciting developments called BlenderBIM

Much more efficient handling of detailed facesets.

Remember, get the latest builds from IfcOpenBot https://github.com/IfcOpenBot/IfcOpenShell/commit/9bcd932bed48486bf5b5f48d24b49329c280462f#comments

Separating elements by their material layers

An advantage that IfcOpenShell has over the many mesh-based viewers is that it is backed by a powerful modelling kernel called Open Cascade. This has already proven its significance for example in the ease of creating 2d floor plans from 3d elements in the SVG exporter. Now, a new feature in IfcOpenShell is introduced, once again demonstrating IfcOpenShell as a forerunner in terms of accurate and advanced visualisation of IFC geometry.

As building professionals readily know, building elements such as walls and slabs can be built up using several layers, for example including brick, thermal insulation and an air gap. This information is typically represented in IFC using an IfcMaterialLayerSet, which describes the thickness of such layers relatively to the axis of the wall. Therefore, it is not a direct part of the geometric representation of the element and therefore not visible in most IFC viewers. Alternatively, IFC exporters can choose to decompose the wall element into several IfcBuildingElementParts describing every of such layers as a separate product.

IfcOpenShell now introduces the possibility to separate the body representation according to its layers, with the appropriate style information applied. Furthermore, topological information that describes how wall end points connect to one another is used to fold such layers around corners where connecting walls meet. To our knowledge this is functionality that is currently not offered in any other IFC viewer, let me know if otherwise.

Currently this feature is in extended testing and is available in a development branch. It will be part of the upcoming new build once testing is complete. Note that it is potentially a time intensive operation and hence will have to be enable with the flag --enable-layerset-slicing in the IfcConvert utility.

Visuals rendered using IfcBlender of the Duplex apartment building model by USACE ERDC and buildingSMART Alliance. 1. A view from the outside with one window removed. 2. An exploded view of the lower level walls showcasing how layers are folded based on topological connectivity with other walls. 3. An image for contrast from a common IFC viewer with only a single surface style per product.

.

Creating 2d SVG floor plans from IFC files

SVG floor plans extracted from the Medical Clinic model by USACE ERDC and buildingSMART Alliance

IfcConvert converts IFC files into a variety of output formats. Most of you will know these formats include geometrical models, either tessellated and easy to understand for importing applications, such as Wavefront OBJ and Collada, or Boundary Representation models in STEP or IGES with support for curved surfaces. Some of you might know the 0.5.0 branch also features an XML output format that contains a decomposition tree and property data of the building elements.

In collaboration with ROOT B.V. a new two-dimensional geometrical output format is added, called SVG (Scalable Vector Graphics). In order to turn the three-dimensional IFC model into 2d floor plans, the model is sliced at horizontal cross-sections. ROOT B.V. uses this format for their web-based tool flatt., an online facility management platform that aggregates sources of additional information into a two-dimensional overview. The development of the SVG conversion has been executed by AECgeeks, a consultancy company founded by one of the main contributors to IfcOpenShell. Both companies are strongly committed to free open source software, so we are happy to see that these developments are merged back into IfcOpenShell for the greater public good.

The SVG output format of IfcConvert is integrated into the public IfcOpenShell source code as of today. The image on the right does not show the direct output of IfcConvert, the isometric projection is added in a vector graphics manipulation tool.

Two new IfcOpenShell releases

It has been a long wait since the previous official IfcOpenShell release. But now, two releases are presented at once, to make up for testing the patience of the community to this extent. On the one hand the new 0.4 release candidate improves on the stability of the 0.4 branch. The 0.5 preview release features more bleeding-edge work. Both releases have been updated for changes in upstream projects and include builds for the new blender 2.73 and 3ds Max 2015 and are built using Open Cascade 6.7.1 (OCE 0.16).

Both releases can be downloaded from the ifcopenshell.org website. As always, it includes IfcBlender, IfcMax and IfcConvert. The 0.5 release now introduces ifcopenshell-python, a Python module for parsing and writing IFC files and creating BReps and explicit triangulated geometry from the implicit IFC geometry. The BIMserver plug-in has been provided with updated executables for its geometry interpretation.

The 0.4 release

An excerpt of the improvements is provided below:

• More faithful use of precision information in IFC
• Fixes and improvements for IfcLShapeProfileDef, IfcCShapeProfileDef, IfcRectangleHollowProfileDef, IfcCartesianTransformOperator3D, IfcSpecularExponent, IfcFacetedBrep, IfcPolyline, IfcPolyLoop
• Prevent errors in triangulations due to zero-length normals
• Fixes for parsing files with comments
• Write unique IDs in Collada files
• Speed up output of Wavefront OBJ files
• Less unnecessary nurbs conversion in the STEP and IGES output
• Check the validity of solids and fillets
• Make parsing and serialization locale-independent

The 0.5 preview release

Simultaneously, a large amount of improvements and new features have been added to the 0.5 branch, which we do not want to withhold from the community. Hence, a preview of the 0.5 branch is released for adventurous users. Most notably, it contains all the improvements to geometry handling as outlined in this post and a brand new Python interface and coupling with pythonOCC. The Python interface has been built in collaboration with the FreeCAD team and its users, which are about to release a new version that includes a very promising full blown IFC importer and exporter. For programmers a new API is introduced that supersedes the IfcGeomObjects API. A new output format is added to IfcConvert which provides textual output from the IFC file that describes the header, decomposition structure and property values in XML format.

Introducing IfcOpenShell Academy

With the introduction of the Python interface to IfcOpenShell and the integration with pythonOCC, IfcOpenShell becomes even more suitable as a platform for academia, research and rapid development. With Python+IfcOpenShell+pythonOCC+Qt a complete framework is presented to interactively and visually present, analyse and construct IFC data. IfcOpenShell Academy provides an overview of quick recipes or building blocks to accomplish common tasks related to IFC. If you wish to contribute a recipe to the cookbook contact thomas@ifcopenshell.org.

Enjoy! And remember that IfcOpenShell still needs your help!

Combining IfcOpenShell and pythonOCC

One of the strengths of the IfcOpenShell library is the vast amount of shape analysis and healing tools that Open Cascade provides. This allows for transforming, validating, generating and analysing geometry in IFC files, for example, to compute missing IfcProperty values, validate geometrical constraints, implement clash- and interference checking and the generation of new geometry.

pythonOCC exposes the rich set of functionality that Open Cascade provides to a Python environment, which allows for intuitive rapid prototyping. Since IfcOpenShell also provides a Python wrapper, the two of them can be easility tied together to create a development environment for working with geometry in IFC files in an interactive, quick, but also, very powerful way.

To demonstrate the possibilities, or rather, the first baby steps towards a more interesting and meaningful demonstration, I put together a quick example here: http://ifcopenshell.org/pythonOCC/example1/. The set of dependencies to get the system running is long, but it is worth it!