Secondary development process of slicer based on workflow of intracranial hematoma

3.1.1. Download of source codes

Because 3D Slicer belongs to open source software, its source codes is hosted on Git Hub. There are two kinds of software, SVN and Git [3], to download the source codes. Among them, SVN is an open source version control system, through the use of branch management system for efficient management; Git is an open source distributed version control system, which can effectively and quickly handle projects. This development chooses Git for codes download and hosting. By downloading Git software and using Git Bush for command line processing.

3.1.2. Compilation of source codes

After downloading the source code to the specified folder, we start to compile the source code [4]. C++ general compiler software is CMake, the use of CMake software for compilation. Because the open source software is through the C++ for the bottom codes, joined the VTK, CTK and other libraries, developed the interface with Python and wrote the module, we should use the same C++ language written Qt graphics processing software interface.

Before compiling, we need to download Qt and install it into the specified folder.

Fig. 1CMake compilation

Enter Qt_DIR in Add Entry and select the Qt folder in the CMake directory under the Qt path.

Fig. 2Configure the QT environment for source codes

Fig. 3Use visual studio to generate SLN files

Click Configure to compile and select MVSC_2017×64. At this point, because CMake is not installed in the same path as Git, the first error will occur. We need to find the PATCH file under Git manually.

Once we find the path, click Configure again to keep it running until the word Configure Done appears. Then click Generate and Open Project in turn, and A. SLN file will be generated and opened with VS2017. Right-click ALL_BUILD and select the generation option. Because it’s the first time to generate, you need to download a lot of files, so the generation time will be very long.

Once generated, a corresponding Slicer.SLN file is generated under the Slicer-build folder. Use CMD of Windows system to open operation. (This process is to configure the environment for the SLN file, which cannot be debugged if opened directly).

3.2.1. Introduction of Qt

Qt [5] is a cross-platform C++ graphical user interface application development framework, mainly used to develop graphical user interface (GUI [6]) programs, but also can develop no interface command line (CLI) programs. In addition to drawing the interface, Qt includes many other functions, such as multi-threading, database access, image processing, audio processing, network communication, file manipulation, and so on.

3.2.2. How to use Qt in visual studio

When we see the 3D Slicer source codes, we will find there are Qt iconic UI files in it. Although Qt is installed, it’s still unable to open UI files in VS. For solving that, we need to search for Qt plug-ins and install them in Tool → Extension and Update. Restart Visual Studio after tool was installed.

3.2.3. Two ways of using Qt

(1) Qt Creator: The tool is writing codes to create the interface. If you read the source codes, you can find that the software has a part, used to write the interface. Especially, for the creation of buttons, slot function writing and so on, this work has a great role.

(2) Qt Designer: The pattern is an interactive visual GUI design tool, which can help developers develop interfaces quickly. It use a way which dragging the corresponding graphics to create the corresponding buttons and layouts, insert fonts and so on.

Fig. 4Use of QT in visual studio

The method used in the secondary development is the combination of the both. Firstly, Qt Designer is used to create the button. At which time, the button is not connected. Then Qt Creator is used to add and connect slot functions in source codes. Finally, the connection operation between button and slot function is carried out in Qt Designer. Because this way is easier. The combination of both simplifies the complex codes written by Creator and facilitates the layout adjustment of GUI [7].

3.2.4. Establishment of 3D slicer workflow

We use Qt to integrate the functions of brain modeling and hematoma modeling, and establish the workflow of craniocerebral surgery in the end.

Fig. 5Establishment of workflow

In the original interface of slicer, only the basic data module and volumes module are useful, so for the medical procedures of intracranial hematoma, the required modules are integrated into the initial interface.

When operating in the original interface, we need to remember the spelling of the name of each module, and we also need to search the module constantly, which consumes time. At the same time, the English interface also has a certain impact on the operation of doctors, so we can carry out the important module of Chinese.

In the original slicer, it takes about 40 minutes to complete the hematoma modeling, image fusion, nerve bundle modeling and intraoperative operation, which is more skilled for slicer users. After the interface is integrated, the modules that need to be used are obvious and arranged in the order of work flow. The preoperative and intraoperative operations can be completed in turn according to the order from left to right and from top to bottom. The completion time is about 30 minutes. For the first time users of slicer, the integrated interface will be more fluent.

Hematoma modeling in the process is demonstrated in Fig. 6.

3.3.1. Partial image processing display

Using the medical image registration kit provided by SlicerElastix [8, 9], rigid body transformation only needs space translation and rotation. In nuclear magnetic resonance images, space can be translated or rotated along $X$, $Y$, $Z$ axes. Therefore, rigid body transformation only needs 6 degrees of freedom. In addition, the toolkit also includes affine transformation and nonlinear transformation. Among them, in addition to the translation and rotation of rigid body transformation, there are six additional degrees of freedom for image zooming in, zooming out and tilting. Finally, the most complex nonlinear transformation has one more local deformation of the image than affine transformation, so it needs more than 12 degrees of freedom.

Fig. 6Hematoma modeling demonstration

Considering the time of image registration, and the medical data used in the research do not have too large image skew and deformation, the fast registration speed of rigid body transformation is selected.

Fig. 7From left to right, Mr, DTI and CT, DTI and Mr registration effect

3.3.2. Rigid registration algorithm

Rigid transformation requires the original image to be transformed into another coordinate system through a mapping, and at the same time, the original image in this coordinate system is consistent with the image to be registered. The general formula is: $B=R*A+T$, where $B$ is the coordinates of the points under the image to be registered, $A$ is the coordinates of the points under the original image, $R$ is the rotation matrix, and $T$ is the translation matrix. The rotation matrix $R$ is described by Euler angle.

Under the description of Euler angle, the matrix rotating about the $X$-axis is:

The matrix rotating about the $Y$-axis is:

The matrix rotating about the $Z$-axis is:

The rotation matrix of Euler angle decomposes the rotation of rigid body into three parts (blue is the starting coordinate system and red is the destination coordinate system).

Fig. 8Demonstration of Euler angle rotation

Step 1: Rotate $\alpha$ the $Z$-axis so that the $X$-axis coincides with the $N$-axis (the Intersection line of the X-O-Y planes of the two coordinate systems before and after the rotation).

Step 2: Rotate $\beta$ around the transformed $X$-axis ($N$-axis) so that the $Z$-axis coincides with the rotated $Z$-axis.

Step 3: Rotate $\gamma$ around the $Z$-axis after transformation to make the two coordinate systems completely coincide.

According to the order of the axis of rotation, this set of Euler angles is called “$ZXZ$ compliance”. So the rotation matrix of Euler angle is $R=R_z\left(\alpha \right)\cdot R_x\left(\beta \right)\cdot R_z\left(\gamma \right)$.

3.4.1. Package with visual studio (VS) [10]

(1) Install VS2017 package plug-in Microsoft Visual Studio 2017 Installer Projects, also in the tool to Extension and Update to online search and download. Restart after installation.

(2) Select the solution, add to build new Project, in other Project type select Setup Project to create. In the pop-up dialog box, we should find the project output and select the main project output item. Then select the main output and create the shortcut.

(3) In solution explorer, we can select the new project and right click to generate. After that, the software is packaging.

3.4.2. Universal software packaging

For the secondary development of software, packaging is the last step. We can choose to generate and package directly from the package project given by the source code. In this paper, we first give a general method of using vs2015 to package [11].

(1) Select the solution, right-click add and choose new project, and select other project types to “visual studio installer” to setup project. Generate the package project under the solution of this project.

In the above figure, “application folder” indicates the files to be added to the application to be installed; “user’s programs menu” indicates the contents displayed in the start menu after the application installation is completed; “user’s desktop” refers to the contents displayed on the desktop (generally, the. Exe shortcut of the application).

(2) Right click “Application Folder” to Add File and select the “.exe” file in the debug directory that has been compiled.

After that, the VS dependency will be added automatically. You can see if there are missing dependencies in the list on the right.

If the dependent file contains a folder, you need to add it manually.

(3) Add. Exe shortcut. Put it in the “user’s desktop” and “user’s programs menu” folders. Select shortcut, you can add Icon, name, description and other operations in the “properties” column.

(4) Add an uninstall program. Copy operation (2), and according to the path “C: Windows / system32\ Msiexec.exe” add to “Application Folder”. Create a shortcut and rename it uninstall and put it in the “user’s programs menu”. Next, you need to indicate the project to be unloaded for exe. Select the item, select product code in the property, copy it into arguments, and add “ / x + space” at the beginning.

(5) If we want to optimize the interface or add some restrictions, you can select add in the view list. Only the default installation path is added here. Select “application folder”, right-click to modify the installation default folder, and then directly modify it to the specified path.

(6) Finally, rebuild the installation project. The generated installation files can be found in the debug or release directory of the installation project.

However, since 3D slicer adds packaged source code to the source code, you only need to right-click generate package to get the files of type exe in the specified folder.

3.4.3. Possible problems and solutions

In The first time, we package failed, that means we lack of NSIS software. After packaging again, we package successful.

(2) After Software packaged, it still had the console. In order to solve the above problems, we have the following operations. First of all, in the CMake, leave the compile type only for Release. If it is no avail, We can add the following codes “#pragma comment (linker, “/subsystem: windows/entry: mainCRTStartup”)” at the beginning of the program. If you want to do it once and for all, you can follow those steps. Step One: Open “Configuration properties” to “Linker” to “System” to “Sub-System” and set to “Windows (/SUBSYSTEM:WINDOWS)”. Step Two: Open “Advanced” to “Entry Point” and set to “mainCRTStartup”. After that, as long as we don't input ourselves, there won’t be a console.

4.2.1. Download of external module source codes

Use Git Hub to find and download the codes, or in the extension module, open the module to be downloaded and find the corresponding source download link.

4.2.2. Compilation of external modules

We can use CMAKE to compile the downloaded codes. Before compiling, you need to let Slicer_DIR in Add List and select the slicer-build folder under the build folder path. Because 3D Slicer is a basic to write and develop the codes of all modules, 3D Slicer needs to be added as the compilation environment of external modules. Sometimes, because 3D Slicer is developed on the basis of VTK, CTK and other libraries, it is also necessary to add VTK-build and CTK-build paths to configure the environment of external modules [12].

4.2.3. Import of external module

After CMAKE compiles, we should Open the Project and right-click on ALL_BUILD next. The external module is based on Slicer, so the source codes also comes with a PACKAGE project, right click INSTALL project or PACKAGE module. After a successful build, a new folder is generated in its root directory. It’s similar to software packaging.

In the open Slicer, we should select this one that is the “Setting” to Modules, and select the import module, import the corresponding files under the newly generated folder in turn, and finally restart Slicer. After that, we import successfully.

4.2.4. Module development

3D slicer supports three types of modules: command line interface (CLI), Loadable module and Script module, but their final interface presentation is no different [12].

CLI is a stand-alone executable file, which is usually implemented in C++. However, the CLI can only do limited input and output, using simple parameter types, without user interaction. In the latest slicer version, slicer can run almost all Python scripts as cli modules by providing interface description .xml files.

Loadable module is a C++ plug-in built by 3D slicer, which can completely control slicer for its operation. The CLI optimizes the complex computation, can use C++ shared library, can realize the complete independent UI.

Script module is written in Python computer language, API can fully access VTK, QT, MRML and so on. It is recommended to use script module for rapid prototyping design and custom workflow development.