Mesh
This page needs attention!
There are many advantages to examining functional data on 3D renderings of the surface of the brain. Most importantly, these renderings help the scientist understand where the different regions are with respect to one another and with respect to the major features of the brain.
To interact with the data in this way, we create meshes of the cortical surface. The process for creating these meshes begins by
- identifying the white matter from T1-anatomical data FreeSurfer
- touching up the segmentations using ITKSnap.
Elsewhere we describe the segmentation process in more detail.
The software architecture for interacting with the 3D meshes is based on a client-server model (mrMesh). The server is a stand-alone program for Windows, Linux, and Mac that uses OpenGL routines for fast rendering of 3D wire meshes. The client is a set of Matlab routines that send commands and data to the server. Although we still create a mesh, smooth and shade them, from mex-functions called from Matlab, the processing has become fast enough within Matlab proper that we could, at some point, eliminate the mex functions and much headache.
To use the mesh software you must follow the set up instructions for Matlab.
On Windows, you must have the libraries and .NET framework installed as described in the (Matlab#Installing_Necessary_Libraries | installing libraries section.) In particular, mrMeshSrv.exe requires two very old runtime libraries, msvcr70.dll and msvcp70.dll. Microsoft never distributed those directly, but there are still some websites that provide them.
On Linux, you can run mrMestSrv.exe using wine (a windows emulator). Some instructions:
• Connect to a Linux machine either via the console or using Chrome Remote Desktop • Pull dev branch of Vistasoft • Sudo apt install wine • Launch mrMeshSrv.exe using wineconsole (it doesn’t work if you just use the regular wine command) • Bumping up the allowed delay in pnet_readmsg (this is actually already checked in to the dev branch
?? Support for the newer Intel Macs is more limited: you can build and smooth surface meshes, but not visualize them (the VTK Mesh Server needs to be recompiled for this architecture).
mrVista analyses are tightly integrated with mesh representations; we do much of our analysis by interacting with the data on a 3D mesh representation of the cortical surface.
We created several means of interacting with these 3D representation.
There is a 3D Window for interacting with meshes. There is also a newer Gray Menu within a volume or gray window which contains a subset of the most commonly used options These two interfaces both allow you to do several things:
* You can build, load and save meshes based on the installed gray/white segmentations. (.class files from mrGray)
* You can display data (corAnal fields, parameter maps) on the surface.
* You can draw ROIs on the surface.
* You can show ROIs defined from other sources on the surface.
It is possible to create, smooth, and display meshes from the Matlab command line interface without invoking the full mrVista interface. The methods are summarized here.
You can use [http://white.stanford.edu/software/downloadsData/mrVistaAnatomy.zip a sample anatomical data set] to experiment with building meshes.
After downloading and extracting the data, find one of the white matter class files (left.Class or right.Class). Currently these files are created using ITKGray-Using; previously we used MrGray-Retired.
To display a mesh, the mesh server must be running (with the command ''mrmStart''). Also, on Windows, you must have Mesh#System_and_software_requirements. To create and visualize a mesh, use the following sequence of Matlab commands:
fName = 'FULLPATHNAME OF WHITE CLASS FILE GOES HERE'
msh = meshBuildFromClass(fName);
msh = meshSmooth(msh);
msh = meshColor(msh);
meshVisualize(msh);
If you would like to add two layers of gray matter to this visualization, you can use the following sequence of commands.
[nodes,edges,classData] = mrgGrowGray(fName,2);
wm = uint8( (classData.data == classData.type.white) | (classData.data == classData.type.gray));
msh = meshColor(meshSmooth(meshBuildFromClass(wm,[1 1 1])));
meshVisualize(msh,2);
This changes the mesh appearance so that it has a more full appearance. Image:MeshGrayVisualize.jpg
To visualize the right hemisphere, you can run
fName = 'FULLPATHNAME OF WHITE CLASS FILE GOES HERE'
msh = meshBuildFromClass(fName,[],'right');
msh = meshSmooth(msh);
msh = meshColor(msh);
meshVisualize(msh);
You can also visualize 'both' hemispheres.
A mesh can be smoothed / inflated either from the Matlab command line or from the window interface described below (Open 3D Window).
The Matlab command meshSmooth() is described above. The default parameter values for mesh creation and smoothing produce a moderate amount of smoothing. The key mesh parameters are
smooth_iterations (default 35) smooth_relaxation (default 0.15) smooth_sinc_method (default 1)
These are set using the meshSet method as
msh = meshSet(msh,'smooth_iterations',30);
msh = meshSet(msh,'smooth_relaxation',0.5);
msh = meshSet(msh,'smooth_sinc_method',0);
A fancier method of inflation uses Jonas Larson's SurfRelax code to inflate (unfold) the mesh while minimizing distance and area distortions. Currently, this code only runs on linux. The steps to use this code from a Windows machine are:
- In mrVista, build a mesh as you would normally, with a smooth level of around 0.2 (that's the default initial value). Save it in the anatomy dir, as usual. For this example, assume the mesh file is named rightNoSmooth.mat.
- ssh to a linux machine and cd to the anatomy dir where you saved your meshes.
- Run Matlab -nojvm, and run: >> mrmInflate('rightNoSmooth.mat',200);
The number of iterations (200 in the example) determines the level of smoothing. Turn it up to 400 for a fully inflated brain. You can also specify a name for the inflated mesh as a third input variable.
This will run and create a new mesh file, rightNoSmooth_surfRelaxed_200.mat, which is an inflated version of your original mesh. Back in Windows, you can load this mesh from the 3D window as you would load any mesh file.
There are some tools to remember the view angle, zoom, and lighting in a mesh, and restore it later. The main tool is, in the Gray_Menu, the '''Mesh Settings''' options. This sub-menu will allow you to store, retrieve, rename, and delete settings for the currenlty-selected mesh attached to the Volume/Gray window. The settings are saved in a file named "MeshSettings.mat", in the same directory as the mesh files. These settings can be used by any meshes in that directory.
With the older 3D Window interface, the menu '''Edit''' | '''Toggle Settings Panel'''. This will cause a UI panel to be added to the left side of the 3D Window, with the same mesh settings options, as well as a manual zoom slider.
For mrVista2 and the mrViewer tool, there is a separate '''Mesh''' menu with these options, as well as a Mesh Panel dedicated to the same functionality.
Image:3D Window with settings panel.png
Other useful command line options:
- To create a montage of a mesh from multiple angles, w/ color bar, use '''meshMultiAngle''' '''Usage:'''
meshMultiAngle(mesh, settings, savePath, colorbarFlag);
'''Example:'''
meshMultiAngle(VOLUME{1}, {'Medial' 'Posterior' 'Lateral'}, 'Images/meshMontage.png', 1);
- To plot the same map / corAnal field from several scans, using the same angle, in a montage: '''meshCompareScans''' '''Usage:'''
meshCompareScans(view, scans, dts, angles, savePath, leg);
'''Example:'''
meshCompareScans(VOLUME{1}, [1:4], 'Original', {'Medial' 'Posterior' 'Lateral'}, 'Images/Comparison.png'), 1);
Regions of interest can be imported onto the mesh from the Gray (or Volume) window.
You can grow a disk region from the cursor location. You can specify the diameter of the desk.
There is a function '''meshGrowROI''' that grows a contiguous patch along the gray matter surface of a mesh, selecting voxels contained within the current data overlay. The patch starts with the mesh 'cursor' (the small set of 3 axes which you can place by double-clicking on the mesh) and grows outward. This is similar to the "grow blob ROI" option in Inplane/Volume views, but is constrained along the gray matter surface.
You can access the menu function '''Gray | Mesh ROIs | Grow Gray Matter Patch from ROI cursor'''.
(This section needs editing).
We describe how to draw ROIs directly onto the 3D visualization.
The visualization has two modes, Interactive and Drawing. Normally, you are in Interactive mode. To draw an ROI, you must enter Drawing mode by typing 'd'.
In Drawing mode, click at several locations to define the outline of the ROI. As you click, lines will appear along the brain surface. This drawing process is slow; if you are patient, you will see an outline appear on the 3D brain representation. To undo the last process type 'u'. To produce a filled ROI, you will want to close the region. You do this by typing 'c'. This will connect the first and last points you clicked, producing a closed polygon.
[N.B. If there are handles/bridges in your segmentation, and in this regions, filling may accidentally spill out across the mesh; in this case, you'll want to double-check your segmentation.]
When you are done, type 'd' again and you will return to the Interactive mode.
If you would like to fill the ROI that you just outlined, type '''d''' (toggles between interactive and drawing mode) and then '''f''' (show cross hair) Click inside the selected region, and it will fill-in. This process (filling-in) takes a (very) long time.
The ROI created this way is not quite standard. To transfer it properly to mrVista, use the pulldown in the 3DWindow | Xform ROI | Mesh ROI to VOLUME (fill layers)
If you would like to draw another ROI, toggle into drawing mode('d'), then the 'delete' key (delete previous ROI) and you can start again.
There are some options concerning how to create the ROI. These are determined in the 3d Window 'Adjust Preferences'. We recommend setting the preferences to layer1 because the update on the mesh is generally more accurate. The higher level layers often touch the CSF and partial volume.
Summary of keystrokes, in a typical sequence:
- '''d''' (toggles between interactive and drawing mode, in this sequence: enter drawing mode click to draw)
- '''[del]''' (delete everything)
- '''u''' (undo last action in drawing mode
- '''c''' (close selected region)
- '''f''' (fill selected region, click within region)
- '''d''' (toggles between interactive and drawing mode, in this sequence: exit drawing mode)
###mrmViewer Stand-alone Mesh Viewer You can use the function mrmViewer to set up a mesh window that you use during a talk. Simply save the mesh from the 3D window in mrVista. Then, before your talk, use mrmViewer to load the mesh and display the mesh. You can close the viewer and even Matlab during your talk. The mesh windows will still be available as long as you leave the mesh server running.
The program brings up a small window that allows you to select the fMRI or diffusion data file. The program starts the meshServer by calling mrmStart, and then brings up a window with the selected data.
- To rotate your mesh, use the left mouse button.
- To zoom, use the right mouse button and move the mouse up or down.
- To pan, use the middle mouse button.
This viewer will display meshes that have been saved from mrVista or image planes and fiber tracts created using mrDiffusion.
mrmMakeMovieGUI: Mesh movie maker
mrmMakeMovieGUI is a tool for making movies (rotating, zoom, translating) of meshes.
This movie maker can be accessed directly or from mrmViewer.
Alternatively, it can be opened manually with one argument: the mesh ID. The mesh ID is the number displayed in the window header of any mesh displayed by the mrMesh server. Sometimes these are displayed as mrMesh 1. In that case the mesh ID is simply the number 1. For example,
mrmMakeMovieGUI(1)
An example of a movie generated with this tool: http://white.stanford.edu/~rfbowen/brainMovie.avi
The mesh code is accessible via the MATLAB command line, and can be scripted. Some useful functions for scripting include:
- '''meshStoreSettings''': saves the view angle, zoom level, and lighting settings for a mesh, in order to retrieve the same view settings later. (E.g., if you want to view several different maps on the brain using the same angle.)
- '''meshRetrieveSettings''': retrieves the stored settings.
- '''meshMultiAngle''': accepts a list of view settings for a given mesh, sequentially sets each view settings and takes a snapshot of the mesh from that angle, then returns a montage image of the mesh from the different view angles. Useful for examining a map from different angles.
- '''pptOpen''', '''pptPaste''', '''pptClose''': accessor functions for calling Microsoft PowerPoint from MATLAB, and pasting images from a MATLAB figure into a PowerPoint file. Currently Windows only.
In additions, there are some example scripts in the repository and on this wiki to show how to take many '''meshPublicationPictures''' is an example script that loads functional data from a mrVista directory and displays it on a mesh. It is useful for creating a series of images for Supplementary Figures, showing the entire data set.
There is also a page containing an Example Mesh Snapshot Script which Rory used to create images of different data maps (retinotopic in this example) across different subjects, hemispheres, and view angles in a consistent way.
(Does this have to be here?)
The mrVista2 package has some tools to project data onto a mesh, even if you don't have your data initialized in mrVista. To do this, you will need:
- an ANALYZE/NIFTI file (or another mrViewer-readable format) of an anatomy segmented in mrGray; ** e.g., anatomy.img
- a mrGray segmentation (.class and .gray files) of that anatomy. ** e.g., Right/subj_rh.class, Right/subj_rh.gray
- an ANALYZE/NIFTI file of the data map to project, coregistered to the anatomy ** e.g., statmap.img
To project the data:
The program will prompt you for the .class and .gray files, and a name for the segmentation (e.g., 'Right').
If you haven't yet built a mesh, select '''Mesh''' | '''Build Mesh'''. If you have a saved mesh, select '''Mesh''' | '''Load and Display Mesh'''.
Set the appropriate thresholds for the map in the mrViewer#Overlay panel.
Select '''Mesh''' | '''Project Data Onto Mesh''' to display the map on the mesh, using the same thresholds and settings.
Make the mesh panel visible ('''View''' | '''Mesh Panel (Ctrl-5)'''), and click on the Prefs button to access the mesh
preferences, controlling how data are projected onto the mesh.
Creating a Flat view from a mesh
We use the flattening tool mrFlatMesh to create a flattened representation from a mesh. This tool was written by Alex Wade.
This tool is usually invoked from the Volume view of mrVista. In that view, the user selects a point in cortex and a radius for flattening. The mesh is flattened and the data from the Volume view are then projected into a Flat view.
The Flat page contains information on performing analyses on the flat view, including using the Flat#Atlas.
(Do we still need this [http://white.stanford.edu/~brian/mri/segmentUnfold.htm segmenting and unfolding page]? I think not - BW)
A mesh is a [http://en.wikipedia.org/wiki/Edge_(graph_theory) graph] structure composed of nodes, vertices, and edges.
mrMesh Server allows you to interact with the mesh outside of mrVista.
This section should contain info about the mesh structure handy for programming with the mesh.
Edit meshGet, meshSet, mrmGet, mrmSet to see more information.
X/Y/Z coordinates are given in "mrVista" space, which for a given brain is about X:0-200 (200 is right-most), Y:0-200 (200 is most posterior), Z:0-150 (150 is most inferior).
'''initVertices''': row 1 = x, row 2 = y, row 3 = z coordinates of each vertex in the mesh. Triads of vertices form triangles. These are the coordinates of the initial, unsmoothed mesh.
'''triangles''': triads of vertices that define a triangle, which is what the mesh surface is composed of. Vertices are specified as indices into the '''vertices/initVertices''' field. Note that they go from 0 to numCols-1, whereas matlab column indexing will be from 1 to numCols. So don't forget to add one if you are going between the two lists. s
'''vertices''': x/y/z coordinates of the mesh in the state you are currently working with it. This may either be nearly identical to or very different from initVertices depending on if you have smoothed or otherwise altered the mesh
'''vertexGrayMap''': provides the node index for a serious of vertices that you can then use to interact with the node matrix at a given vertex. Make sure you '''recompute vertex/gray map''' in order to fill in this field correctly. The nodes and edges can be accessed through the Volume#The_MATLAB_VOLUME.7B.7D_variable.
'''normals''': specifies a vector normal to the mesh surface at that location (information about the curvature of mesh)
'''colors''': these are 4xnumVoxels matrices where the first three rows are the R, G, and B levels, and the fourth row is the alpha or transparency level (128 is full opacity). This field is updated to display ROIs, activation, fiber endpoint, etc on the mesh surface, usually using the '''meshColorOverlay''' function.
- Has two types of sulcus colors: [96; 96; 96; 128], [160; 160; 160; 128];
- Has as many other colors as your ROI does. E.g., purple: [128; 0; 128; 128]; % purple is equal parts R and B, no G
- To change what the mesh looks like, you can manipulate the the colors matrix, and then update the colors field: '''mrmSet(msh,'colors',uint8(round(newColors')));'''
==How is a mesh created?==
- '''mrgGrowGray''' turns the voxels from the gray/white segmentation created by itkGray into a gray matter graph structure with a specified number of gray layers. It returns nodes, edges, and classData. * A gray voxel is equivalent to one node.
- '''meshBuildFromClass''' creates the mesh data structure ('msh')using the mex-file build_mesh, which takes input arguments of all the white matter voxels, the mmPerVox, and the hemisphere. *We can get and set information from various fields in the 'msh' data structure using the functions '''mrmGet''' and '''mrmSet'''.
- '''meshVisualize(msh)''' opens a tcp/ip server window and displays a visualization of the mesh. However, we often want to manipulate the mesh in various ways before visualizing. * For example, we can use '''meshColor(meshSmooth(msh))''' to view a smoothed mesh.
'''mrgGrowGray''' returns nodes and '''meshBuildFromClass''' returns msh.vertices. However, the mesh created does not have a vertexGrayMap mesh field. This field is important for connecting the vertices to the nodes. *You need a vertexGrayMap provides the node index for a serious of vertices that you can then use to interact with the node matrix at a given vertex.
- nodeIndex= msh.vertexGrayMap(1, vertexIndex); Computing vertexGrayMap:
'''Note:''' mrVista only computes the vertex gray map between nodes with functional data and vertices, which creates a problem whenever the user would like to get the coordinates for points outside of the functional data.
mrManDist(grayNodes,grayEdges,startPt,dimdist,[noVal],[radius]): Computes surface space between points on mesh gray matter.
'''grayNodes''': An 8xN matrix with N signifying the total number of nodes.
*Rows 1-3 specify the (coronal, axial, sagittal) slice numbers for each gray matter node, respectively. (Note that this order of rows is different from the convention for other mrVista strucutres, such as the coordinates saved in the coords.mat file of the Volume views. If you have an open Volume window, then the field VOLUME.coords([2 1 3],:) will correspond to the first 3 rows of grayNodes.)
*Row 4 specifies the number of edges connected to this node.
*Row 5 has edge information. Neighbor nodes are connected by edges, and edge index can therefore give you neighbor information.
**Neighbor nodes are directly connected on the mesh surface, but not necessarily in Euclidean space/Volume space.
**The value in Row 5 indicates the starting index for determining the neighbor nodes. For example, given a row 4 value of 7 and a row 5 value of 21, the neighboring nodes are from index 21 to 27 in the adjacency array.
*Row 6 has information about the gray matter layer for this node.
See the Volume#The_MATLAB_VOLUME.7B.7D_variable for more information on the format of the '''grayNodes''' variable.
Other helpful functions for meshes: *meshCursor2Volume *mrmMapVerticesToGray
Some of the information on this page is out-of-date. In particular, we do not use the .class files any more.
[http://web.stanford.edu/group/vista/cgi-bin/wiki/index.php/Software#mrMesh Original Mesh page]