Neuroengineering IGERT Session 3
Sep 5, 2023
Neuroengineering IGERT Session 3
David Van Essen, WashU
I encourage you to sign up to tour one of our imaging,
or both of the imaging centers
that are being shown around today.
I'm gonna pass it over to Professor Kahn,
who's going to introduce Professor David Van Essen.
Thanks so much.
Alrighty.
So it's really a great pleasure for me
to be able to introduce David Van Essen to you.
David is the Edison Professor of Neurobiology
and the head of the Department of Anatomy and Neurobiology
at Washington University.
Very, very short little background.
His degrees are from Caltech and Harvard Med School.
And then he went on to do post-doc work
at Harvard with Hubel and Wiesel.
The neuroscientists among you will know
that Hubel and Wiesel went on to win the Nobel Prize
for their work on the physiology of the visual system.
And a good portion of David's work
continues in the visual system.
David has been the Editor-in-Chief
of the Journal of Neuroscience,
the President of the Society for Neuroscience,
the Founding Chair of the Organization
for Human Brain Mapping, a busy fellow.
He's a fellow of the, speaking of fellow,
he's a fellow of the American Association
for Advancement of Science,
and has won any number of awards.
I won't bore you with them, but it's very impressive.
Any of you who have ever seen
the justifiably famous wiring diagram of the visual system
with no fewer than 32 different visual cortical areas
and incredibly elaborate
and hierarchically organized connections among them
have David Van Essen to thank.
Blame.
Sorry?
Blame.
No, blame, there you go.
And that kind of approach to a system,
what are the regions and how do they interconnect
and how does that whole elaborate connectivity
give rise to the properties we care about
leads to his current post and his current passion.
Post is Director of the Human Connectome Project,
and that passion, if I may be so bold,
is the amazingly ambitious goal
of really basically providing the wiring diagram
of the entire cortex, if not the entire brain,
characterizing brain connectivity,
how it varies across individuals,
and how that relates to cognition and behavior.
Amazingly ambitious, amazingly important,
requiring the development of new tools
and the synthesis of an amazing amount of information
that plots the course for the future of our field,
if I may be so bold.
So it's a great pleasure to introduce David Van Essen.
Thank you very much, Neil,
and thanks to the organizer for giving this opportunity
to talk to you about what is indeed my passion
and obsession for the moment and for some years to come,
the Human Connectome Project.
Let me try to put it into context
by reviewing some important factoids.
So each of you has about 1,500 grams worth of tissue
containing not quite 100 billion neurons overall.
The dominant structure is the one we see here,
the cerebral cortex, 80% of your brain mass,
but only 20% of the neurons.
It's got the great majority of the synapses,
150 trillion or so, and the wiring in the white matter
is about 100,000 miles worth by some estimates.
The cerebellum is not to be forgotten,
even though it's only 10% of the brain mass,
it's 80% of the neuronal number,
thanks for all the granule cells.
What's keeping you alive and breathing
and your heart beating and all
is the rest of the brain subcortical
and brain stem structures.
Only 8% of the brain mass and less than,
fewer than 1% of the neurons, but getting their job done.
So the broad question is how do we untangle this mess
because it's not just the neurons,
it's all the connections we need to decipher.
And some years ago in 2005,
Olaf Svorens coined the term connectome,
building on omics, proteomics, genomics, et cetera,
to articulate the notion of a comprehensive map
of neuronal connection would be the connectome concept.
I insistently keep the quote marks around this
because comprehensive really has to be looked at
at multiple scales.
It is not feasible in my lifetime,
and probably not in yours,
to get a total count of every synapse
and every neuron in a human brain.
But for sure, our immediate focus
needs to be on multiple domains.
So the concept of a macro connectome
is looking at the long distance connections
between gray matter regions through the white matter
at the level of resolution afforded by imaging methods,
which is basically a millimeter or a little more.
But let's pay respect to the importance
of the micro connectome,
which is the idea of indeed charting
the location and connectivity of every single synapse,
of every neuron, and also all the glial cells.
But that can at best be done in pieces of tissue
that have yet to reach a cubic millimeter
that are not even up to the size
of a conventional brain voxel.
There is also a third world
that I like to call the mesokinectome
that I think will be very important in the long haul,
which is using more conventional
but increasingly powerful tracer methods
to trace local and long distance connectivity,
but it can only be done a chunk at a time.
It has to be done over many different animals
to build up an overall picture.
So with that as a broad perspective,
we will focus for the rest of my talk
on the macro connectome and mostly about the human brain
and how we can capitalize on some impressive recent advances
in noninvasive MR imaging.
But before we do that,
I want to lay the neurobiological groundwork
by putting some important neurobiological complexities
out there on the table
because they are really tough problems to wrestle with
and while there's progress,
there are also serious compounds.
So I'll first say a bit about monkey business,
then the human brain,
and then a lot of time on many pieces
to give you a progress record of where we are
and where we're heading for the human connectome project.
So the complexities that I want to expose
and discuss are the cortical convolutions,
the challenge of parcellating the convoluted cortex,
recognizing what the reality is of brain connections,
coping with the realities of individual variability,
and the technical aspect
for diffusion imaging tractography studies
dealing with what's in a fiber bundle.
In the white matter.
So back to cortical convolutions for starters.
So I started in this field back in the early 70s
and was very quickly frustrated
by the conventional approach of taking a monkey brain
or another experimental animal
and slicing it up literally
and then publishing results
in a series of slice by slice representations
that are informative
but it's very hard to build up a big picture
of what is the overall organization of data
on the convoluted cortex.
So computers were nowhere really on the immediate horizon
for helping solve the problem
so I ended up inventing a manual method
of making cortical flat map
for initially a little piece of cortex
and then John Matzel and I made the first monkey flat map
for the entire cerebral cortex.
That was a starting point.
It was also frustrating
because it was very tedious
and didn't scale at all well
but we had to wait until the 1990s
before MR structural imaging came to the fore
and then we and other groups
developed segmentation algorithms
that could automatically capture the shape
of the cortical folds in a segmented volume
and then let the slice,
I mean the slices be sutured
into an explicit surface reconstruction
that could be inflated and flattened
and used as a representation for many kinds of data
and one of the important kinds of data
is the arrangement of subdivisions of the cerebral cortex
into component areas or parcels
based on whatever information was available
and so we started with keying on architectonic
and connectional and topographic
and functional distinctions
and saying that when Dan Feldman and I
looked at this 20 years ago,
we could find evidence for several dozen visual areas
plus a bunch of other areas in the cortical mosaic.
That was one stab at taking the data available at the time.
In subsequent years,
there have been an additional sets of parcelations
that have not only been published
but have been put on surface maps
and can be registered to a surface-based atlas
that you see in this flat map visualization here
and it's been a continuing frustration
that there's no real consensus on the field
as to what are the parcels
that everyone can agree on.
We still aren't there.
Last year, we published a composite parcelation
from three of the, what I think are the more reliable
architectonically generated parcelations
and the upshot is, by this best guess,
there are probably around 130 to 140 cortical areas
in macaque neocortex, but that's a soft number
and certainly not all investigators would agree with that.
That's the right ballpark, I think.
Importantly, this is not a cookie cutter kind of parcelation.
There is literally a hundredfold range
in cortical surface area between the tiniest areas
like an orbitofrontal cortex and the giant area B1,
itself 10% of the cortex.
So this is part of the anatomical reality for the macaque
and the other part of the reality
has to do with connectivity.
So as you heard in the introduction,
there was evidence two decades ago
for literally hundreds of pathways.
It seemed very complex at the time.
This is a way over simplification of the reality,
as you'll see, but we published not only
this wiring diagram in the way you see here,
but we also, in the same paper,
published what we called a connectivity matrix
and what I would now also equate
to a parcelated connectome, 32 by 32 matrix saying,
is there a connection or not?
So this is just a binary version,
but the concept of encoding connectivity
via matrix representation has that kind of history.
And now it can be expanded because there have,
in ensuing years, been various publications
reporting not only the presence of more connections,
but now finally the emergence of quantitative estimates
of connection strength.
And there's one published study
and one that we hope will be accepted soon
from Henry Kennedy's group in France
using classical retrograde tracers,
but carefully quantitating the number
of retrograde labeled cells in each cortical area
and then expressing that as the fraction
of labeled neurons from, in this case, V1,
receiving inputs from 25 different cortical areas.
So the first point is there's even richer
area to area connectivity than had been appreciated,
a couple of dozen for V1.
The second point is that there's an enormous range
in connection strengths, literally five orders of magnitude
from the massively powerful V2 back to V1
to moderate to very weak,
but potentially still meaningful connections.
Also, when plotted on a semi-logarithmic plot,
you can see that they follow
a systematic log-normal relationship.
And that turns out to be true
for every area that's been examined.
Another point is if you look at the connections
of V1, V2, and V4 on this cortical surface map
with a logarithmic scale here,
you can see the spatial pattern is complex.
The connections are predominantly from nearby regions,
but sometimes at a distance.
And you can see that the profiles are different in detail,
but there's a lot of similarity when you view it like that
or when you view the connections of V1 in green,
V2 in blue, V4 in red.
They differ mainly in the strength of connections
more than the presence or absence.
So this is an important reality
for what we know about the macaque.
And it is likely that similar principles
will extend to human cortex,
but it's likely to be even more complex.
And even in the macaque, some of the frontal lobe areas
that this second study that's been,
is in review, reveals in frontal cortex,
there may be 60 or even 80 or more cortical areas
projecting to one particular target.
So it is not simple, and we just have to cope with that
and factor that into how we're going to interpret
human neuroimaging studies.
So as we move now to human cortex,
the first complexity is not just that the human cortex
is much more convoluted than macaque cortex.
It's also much more variable.
So if you look at lateral views
of a half dozen healthy young adults,
you can see the folding patterns differ in detail.
You can appreciate some of those differences a little more
if you look at one region versus another.
So everybody has a central sulcus,
and everybody has a superior temporal sulcus,
but in the posterior aspects of the superior temporal sulcus
it's so variable it's almost like it got randomly crumpled,
and that's again part of the reality.
And we can quantify the regions of consistency in blue
and the regions of high variability in yellow and orange
by different measures, but it also raises the question of,
well, first of all, why is it variable?
And secondly, what other things co-vary,
or at least are part of this overall picture?
So I like to distinguish five major types
of individual variability
that we need to have in consideration.
From the connectome standpoint
that I'll be getting to shortly,
some of them are kind of just noise.
They're confounds rather than profoundly
of intrinsic interest.
Two of them are ones we really care about.
So the confounds are everybody's skull is a different shape
and everybody's brain is just different
in overall dimensions.
Then secondly, as we've seen,
the cortical folding patterns differ dramatically.
And where the cortical areas are relative
to particular folds is also distinctly variable.
So those we have to cope with.
What we care about is every person in this room has,
for instance, an area V1,
but there will be threefold or so variability
in the surface area of one person's V1 versus another.
And that might have, presumably,
has some interesting differences in how we perceive.
And ditto for every other cortical area
that shows at least twofold variability
when it's been looked at.
And secondly, we think differently, we act differently.
It's because our brain circuits, the wiring is different.
And some of that is surely reflected
in the long distance wiring that we want to gain access to.
A quick sidebar to something I think helps put a context
into why there's so much variability
in the folding pattern.
It starts with the observation
that folding occurs late in gestation.
So this is a preterm infant
at just beginning the third trimester.
And by birth, the convolutions
have largely established themselves.
It's still 1 3rd, the adult brain side,
but the bulk of the folding is all set up there.
This turns out to be the time
that the white matter is growing massively
and the long distance cortical-cortical connections
are getting themselves pulled together, as it were.
And speaking of pulled together,
the idea that I promulgated 15 years ago
is that there's literally mechanical tension
that is physically generating force
through these long distance fiber bundles,
that is mechanical tension in axons
could be the force that drives,
pulls strongly connected regions together
and weakly connected regions
allow a sultus to form in between them.
It's an attractive hypothesis.
Neurons, in fact, generate tension
if you look at them in tissue culture.
It's an unproven hypothesis
and there are competing views,
but what I would say is it's also a unifying hypothesis
because this one mechanism
can account for the consistency of folding
where it does occur if that is driven by large areas
that have consistent connectivity patterns.
And the highly variable patterns
that I've already touched upon
could be regions where lots of tiny cortical areas
that have a lot of variability and connectivity
are essentially balkanized forces
with different outcomes in each individual.
With that in mind,
whatever the source of the variability,
if we're going to compare across individuals,
we have to have good ways for compensating
for individual variability in the folding patterns
and comparing one region versus another
with some confidence that they are,
as best as we can manage,
representing corresponding locations.
And that leads me to the very important idea
of brain atlases as a common anatomical substrate
for making comparisons across individuals.
So if you take one individual MR scan slice
versus another, the folding pattern is different.
There have been a variety
of volumetric registration strategies.
Most recently, nonlinear registration
such as FSL's FNRT is how it's pronounced.
This is a nonlinear volume atlas
that achieves excellent registration
of the subcortical structures
and so-so, better than some but not great registration
of these convoluted regions of cortex.
To register the cortical regions optimally,
we need to have a surface-based approach.
And the best one currently available
starts with the FreeSurfer method
of not only generating the surfaces
but registering them through
a spherical nonlinear registration paradigm
to generate an atlas.
And we, for the Connectome project,
have taken it one stage further
from FreeSurfer's independent left
and right hemisphere atlases
to a pair of left and right hemispheres
that are identical in shape
to what FreeSurfer generates
but have been registered to a common atlas target
so we can identify corresponding locations
in the left and right hemispheres
and as you'll see, be able to explore
hemispheric symmetries and asymmetries
using this approach.
So with that in mind,
what I would say is there has been progress
in generating progressively more accurate
surface and volume-based atlases
but they are not perfect.
They still don't achieve precise registration
particularly in some regions of high variability
and so atlases will continue to evolve
and I'll touch upon one of the ways
that can happen towards the very end of the talk.
But with these atlases, we can also paint them
with parcellations of human cortex
just as we did for the macaque cortex.
So there's the classic Brodmann parcellation
which is still widely used today
because there's no other atlas
that is surface-based and comprehensive
with regard to staking out a claim
for nearly all of cerebral cortex
but the Brodmann atlas is seriously inaccurate
in a number of places or incomplete
especially because it's too coarse-grained.
The German group of Zillis and Amunds et al.
have generated a set of cytoarchitectonic atlases
that have been accurately registered
to the cortical atlas by Fischl et al.
and we can have probabilistic atlas representations
that encode some of the variability
that persists in these maps
and then we've taken it a step further
and generated population average composite maps
for a set of architectonic sub-parcellations
and also retinotopic parcellations from published studies.
And what you can see here is a current state-of-the-art
that has slightly over 50 surface-mapped areas
which in aggregate cover about a third
of human cerebral cortex.
And that leaves the majority of cerebral cortex
for which claims have been staked
but there have not been accurate surface maps generated
and that leaves a large lacuna
that is going to be filled in
by progress from the Human Connectome Project
as well as other efforts that you'll hear about.
So finally, let's shift gears
and jump into the Human Connectome Project proper.
A quick historical reflective,
it was only 20 years ago
that Nobel Laureate Francis Crick in it
and Edward Jones bemoaned
the backwardness of human neuroanatomy,
not only the lack of data
but the lack of methodology at the time
for exploring human brain anatomy and connectivity.
They didn't appreciate in this brief article
that the seeds had already been planted,
the functional MRI had been born the year before
and in fact, an amusing comment in an article,
the first fMRI pair of articles,
Kumail Ugrabil pointed out
that they saw resting state fluctuations
that might be informative about brain connectivity
and that lo and behold has led to a cottage industry
that we'll be talking about.
So urgently needed methods have developed
and evolved over the past two decades
and several years ago, NIH made a bold commitment
to announce a competition for the Human Connectome Project
and the objective is to chart the macro connectome
and its variability in healthy adults
and the Wash U in Minnesota and eight other institutions
got a $30 million grant to get this effort underway.
There's also a parallel grant by MGH
to develop what they've already succeeded
in a customized MR scanner
that is also providing value added to this effort
but I'm gonna focus on our WOMEN
as we call it Connectome Consortium
that has 10 institutions and all 90 people
working on various aspects of this complex
and exciting project and let me just run quickly
through our overarching objectives.
Our target is to study 1200 healthy adults
drawn not off the streets of St. Louis
but rather from a population that we have access
to individuals born in Missouri
and where we can identify twin pairs
and for added analysis power,
their non-twin siblings in a way that gives us
the greatest information about heritability
and genetic analyses.
As you'll see, we're using cutting edge
neuroimaging methods that involve as the core
scanning device a single 3T SCIRA from Siemens
with a customized gradient that just arrived in Wash U
and has been handed over to us just yesterday
when it got tuned and ready to go.
200 of the subjects will be scanned in addition
at seven Tesla in a scanner that's at Minnesota
and if we're lucky, the 10.5 Tesla scanner
that they're currently putting into play
will be ready for action before the project is over.
We'll acquire diffusion imaging analyzed
using tractography methods, resting state fMRI,
task fMRI, you'll see examples of these.
In addition, 100 of the subjects will be scanned
with MEG and EEG at St. Louis University.
We'll have extensive behavioral testing
on all 1200 subjects.
We'll acquire blood samples and in year five
do genotyping that we hope will be affordable
at full genome sequencing if the price continues to plummet.
And then all of these data will be made freely available
on a targeted quarter release strategy
and it won't just be dumped into a giant repository,
it'll be made available in a user-friendly
informatics platform that I'll say a bit more about
and the idea is to let the field as a whole
do some serious data mining and discovery science
for whatever's of interest in this rich data set.
But in general, it will relate brain connectivity
to individual capabilities and differences
and link it to heritability
and in due course to genomic analysis.
And then NIH didn't do this
just out of their intellectual curiosity,
it's obviously driven by a hope and expectation
that these data will provide an invaluable baseline
for future studies of any number of brain disorders
as well as neurosurgical and developmental
and aging related questions.
We are just about two years into this project
and we have one more month of intensive refinement
and optimization.
The hardware is fully refined,
pulse sequences are continuing to be tweaked
and a whole lot of analysis methods
and development of the informatics platform are underway
and some of those will continue
but in August, we will launch
what we call the production phase two
and aim to scan 400 subjects per year
and then we're also committed to keeping these data
available through the rest of this decade.
A word about the informatics structure
because that's near and dear to my heart.
We estimate that there will be a petabyte's worth of data
that's worth keeping because we want people to access data
at many different levels.
It'll be placed on the ConnectomeDB database
that is built on the XNet platform
that Dan Marcus has grouped.
Much of the processing will be done
on a high performance computer
that we were fortunate to acquire through Fred Pryor's work.
You'll see examples of the visualization
and analysis platform Connectome Workbench
that my lab has spearheaded the development of
and as I say, we wanna make the data available
at all levels from quick access
to very compact, parcelated Connectome data
that you can just data mine and explore on the fly
to some levels of access to much richer, more informative
but more complex data sets
and those who want early stages of minimally processed
or even the raw data, we wanna make that available.
You pay for the box, we'll send you a Connectome in a box
basically in a hardware storage device.
Let me illustrate some of the advances
that have occurred on the front end of this
because they're not only important
for the Connectome project
but we hope they will percolate out
into the field more broadly.
So conventional multi-slice imaging
is acquire a slice at a time
and then it typically takes many seconds
to capture data from an entire brain volume
especially if you're aiming for high resolution.
So Camille Louberville's group at Minnesota
has spearheaded the development of
what's called multiband imaging
and the basic idea is to excite multiple slices.
Here you see four but we're going up to multiband eight
for the fMRI imaging.
Excite the slices simultaneously
and then more quickly cover the entire brain volume
and then use, in our case, a 32-channel head coil
to work backwards and interpolate
what is the signal coming from one slice or another
through a matrix inversion process
that lo and behold works impressively well.
And I'm getting ahead of myself methodologically
but this is a resting state fMRI analysis
and basically from Steve Smith's et al.
analyzed the standard acquisition
that has 2.5 seconds per brain volume or TR
and when analyzed using the accelerated four-fold
or nine-fold, basically there's a big gain
in the signal to noise, pictures look prettier
and estimates of the signal to noise
for the baseline normal acquisition in blue
versus the accelerated shows a robust improvement
in performance and we'll come back to the methodology
that underlies that.
But first I want to say a bit about where we are
with structural connectivity.
First of all, scoping out the problem,
the white matter of the brain on the good news side
we know from just literally classical brain dissection
anatomy that is organized in fiber bundles
that in many cases, many regions are organized coherently
into fascicles that are aligned with one another.
But bear in mind that each one of these fascicles
contains literally several hundred thousand axons
per white matter bundle and they aren't always going
in parallel with one another.
There are any number of places that even
the classical anatomists recognize
that there are crossing fibers
because there's a lot of traffic getting
from each cortical area to all of its targets.
So to attack this kind of problem,
the fundamental anatomical imaging method
is diffusion imaging in which you take
a structural image for starters
but then the diffusion imaging sequence
is able to estimate the diffusion that is faster
in some directions than in others
and one can get a map of diffusion anisotropy
just along the several axes and color code it
to show the dominant fiber bundle orientation.
But if we want to capture not just the dominant fibers
but also the crossing fiber matrix,
we have to use high angular resolution.
We'll be gathering data probably
from about 500 different directions or angles
and then using sophisticated reconstruction methods
to estimate fiber orientations for the dominant
and secondary and in many cases
the tertiary fiber bundles as accurately as we can.
To get these data as high as quality as possible,
we first of all are using a customized scanner,
I alluded to before.
It has a number of hardware features
that give us 100 millitesla per meter
gradient strength maximum
and that's two and a half times
the standard Siemens scanner
and empirically we're finding it's a big win
in terms of the quality of diffusion imaging data
that we can acquire.
In addition, these multiband acquisitions
that I'm alluding to give us a faster frame rate
and allow us to get more data per unit time
and higher quality data.
So we expect to be getting voxel sizes
of about a millimeter and a quarter
which is about four fold better volume wise
than the standard two millimeters
that has been state of the art
as well as better methods for analyzing fiber orientations
and their dispersion.
And then once we get these kinds of data per voxel,
we have to not connect the dots
but connect the lines, the oriented fiber
and that is the process of tractography
and that's also a rapidly evolving field
and I wanna just touch upon
a couple of the methodological issues.
There are two widely used methodological approaches
to tractography.
One generates by far the prettiest pictures to date.
It's called deterministic tractography
and it generates these colorful images
that show the dominant fiber trajectories
from one location to another.
The problem from my perspective
is it captures only a small fraction
of the anatomical reality
that I took pains to emphasize
when talking about the monkey business.
We know each location in cortex
is connected with many dozens of other cortical areas
and the methodology, if it fails to capture that,
the reality
is the data quality is of course one rate-limiting step
but also the analysis strategy
and we're, in our consortium,
focused mainly on what we call probabilistic tractography
that's been developed by Tim Behrens and others
in the Oxford group
and is part of the FSL software suite
but we're continuing to refine it
and it can already estimate multiple fiber trajectories
to give you an estimate of the regions
of one gray matter to another gray matter region
that are likely to be connected in a probabilistic
rather than deterministic sense.
We're also adding to this equation
the ability to map connections
to cortical gray matter
and use surface representations to show, for example,
a early version, very preliminary data
of probabilistic tractography
from one location to various others
and a different location
has a different pattern of connectivity
and we wanna improve the analysis
as well as the visualization methods.
One of the challenges
and one of the things, by the way, I'm trying to do
is as we go along, emphasize some of the challenges
that have a neural engineering underpinning.
We need better algorithms by individuals
who are well-suited to take on one or another
of these deep methodological challenges.
So, in brief, we know that in a blade of white matter
that feeds up into any particular gyrus
that the fibers are fanning in and out
and feeding into neurons coming from
or going to regions all along both banks of the gyrus.
What tractography algorithms currently see
is a huge bias in favor of the gyro crowns
because that's just the way that the data are dominated
and the analysis method helps foster
and so we get very uneven estimates
of the density of inputs across the cortical mesh.
We need better data, but we need better analysis
that's based on the actual anatomy of fiber trajectories
rather than just this schematic
and this is a whole domain
that is going to be ripe for further exploration.
But let me move on to another domain.
I want to get to functional connectivity,
but I want to introduce it using a visualization
and analysis approach that capitalizes
on the Connectome Workbench software
based on Carrot software my lab has developed for decades,
but we actually started rewriting that book from scratch,
recoding the entire software package
to get it cleaner and better laid out
and earlier this month,
we just released Connectome Workbench 0.51.
You can download it, the software,
and pilot test data sets in a tutorial
capitalizing on a lot of work over the past couple years
to develop the platform to this stage.
It's got a lot of nice visualization features.
We're basically trying to configure ourselves
for efficient whole brain analysis
fitting the human Connectome endeavor
so we can have one window
that shows whole brain cortical views,
left and right hemispheres
from both lateral and medial aspects.
You can spin them around and zoom them
and in another side window,
you can have the brain volume visualize
all the subcortical slices in a montage
that lets you see as much of the brain
as you're interested in
and we can overlay the FreeSurfer peel
and white matter surfaces of this individual
to see how well the surfaces are aligned
to the underlying anatomy
and then we can start analyzing data
but one of the things we discovered
given our interest in looking at surfaces
and subcortical volumes in a coordinated coherent way
is it was unattractive to use
the current available at the time
surface data sets separate for left and right hemisphere
and volume data sets
rather given the complexity of connectivity data
that we have,
we decided it was worthwhile investing
in developing and putting out for the field
a new data format that we call the SIFTI data format
and it really is rooted in the concept of brain ordnance
where for the cerebral cortex,
the natural geometrical representation
is a surface mesh with surface vertices or nodes
for the left and right hemisphere
and then for the subcortical structures,
we don't want to represent the entire brain volume,
we want to focus on the gray matter voxels
within known subcortical structures
and so we have this composite representation,
a SIFTI file which can represent all of gray matter
in a coherent coordinated way
and one brain gray matter volume
has about 90,000 what we call gray ordnance
and for the tractography,
we'll have a parallel domain,
a SIFTI file that is just the white matter voxels
and this is a sensible way for parsing the data
to reflect the underlying information that is of interest
and so we can take for example
and I'll show you a movie in a moment
but I'll set the stage for it by saying
if we want to take a time series data
of the brain fMRI or bold signal,
we want to represent it as gray ordnance
as a function of time for the left and right hemispheres
and their subcortical structures
and if we take one particular location,
it will have a particular time series
and another location will have a different time series
and a vertical slice one moment in time
will cover the entire set of gray ordnance
and this format turns out to be useful
not only for time series data
but for a variety of other data formats
such as the task fMRI contrast
that I'll be showing you in a few moments
but with that as a quick algorithmic
or representational introduction,
let's look at some pretty pictures.
So this is one slice in the moment of one subject's time,
the bold activation signal with global signal regress
but basically it's a measure of brain activity
or the fMRI signal accessible at that moment in time
across the left and right hemispheres
and in the subcortical structures
and then we can in fact watch a movie of that
and see it change from moment to moment
over a brief period of time on the brain surfaces
and also the brain surfaces map back into volume slices
for added value.
So that's one quick movie.
Let's take another look at the same subject
looking just at the left and right cerebral hemispheres
and get a better idea of what the dynamics
we're actually dealing with
when we talk about resting state functional connectivity.
So this is again a movie of 40 seconds
in the life of this individual
and you can see there are regions of high activity
that tend to be correlated roughly but not in detail
and regions of lower activity
and they fluctuate over these very slow
but very interesting time courses
and this is the input data
for doing resting state analyses
and there are many ways to do these analyses,
independent components analyses.
Jack, I don't know if he'll talk about resting state
but he'll talk about other ways
for analyzing fMRI data series
but we want to take these time series data
and use the approach of generating functional connectivity maps
for all brain gray ordnance
correlated with all other brain gray ordnance.
So this is essentially the concept of what we call
a dense connectome,
a functional connectivity matrix
for all 90,000 by all 90,000 gray ordnance
and then we can make maps for one brain location.
Here is the functional connectivity map
generated from this time series data.
Another brain location
has a very different functional connectivity map
and we want to be able to probe these
capitalizing on the high spatial resolution
that are available in our acquired data sets.
That poses a challenge
from the fact that 90,000 by 90,000
gives us about a 30 gigabyte data set
for what we call the dense connectome
of one subject for one modality
and that's not something you want to go shipping
over the internet willy-nilly
so rather we've developed an infrastructure
that allows the data set to reside on our database
but to probe it one slice at a time
that is one functional connectivity map at a time
and view it on your home platform.
So here we can literally,
I mean I have it set up
but I'm not going to take the time
to show you it in real time.
You can click on a location
and see a functional connectivity map,
click on another location
and see nearby the functional connectivity is very similar
and then probe another location
and see a totally different functional connectivity pattern
and access to that information
gives you the ability to probe functional connectivity
at high spatial resolution
and then there are other kinds of questions
one can start to probe
such as network dynamics.
The correlation analysis I showed you before
kind of looks at cross correlation
across the entire acquisition time
which for the connectome project will be a full hour
but we know that brain states
actually change from moment to moment
and I'm going to whiz by both the concept
and the first realization and analysis
that Steve Smith has led
to try to go beyond the assumption
of static brain networks
to the idea that at one moment
sub-networks may be strongly interconnected in one way
and in another moment
they may have a different set of interactions
and we'd like to capture this
by analyzing the data
and picking out what Steve has called
temporal functional modules or modes
and look at how those dynamics change
over an acquisition period
and I'm not going to try to explain it in detail
I'll just say, whoops, there was a nice movie
that may or not, but anyway
you can imagine
we can get a frame by frame representation
of fluctuating networks and sub-networks
and I'll just fast forward
but I hope you get the flavor of it.
We have access to rich temporal dynamics
that there are going to be a variety of ways
to explore, to try to decipher
how brain states change from one moment to another.
Now I'm going to whiz through three other modalities
each informative, but just to give you the flavor of them
task fMRI, we will acquire data
using about 10 different tasks acquired
over the course of an hour
and examples of that here
is a population average data set
for subjects viewing random motion dots
and here is the same population
viewing structured motion, biological motion simulation
if you take the difference of them
you get a signal that is emphasizing
the specialization for biological motion
we can take a different task
and look at face activations
or in this case place activations
in medial aspects of the hemisphere
we can have more fancy cognitively oriented tasks
such as a social cognition
kind of a theory of mind task
that activates interesting regions
of lateral temporal and lateral frontal cortex
working memory activates its own complex domains
so this is the flavor of the task activations
and the kinds of analysis and visualization options
that will be emerging from this component
of the connectome project
we'll spend two hours or so with each subject
capturing information about their behavioral capacities
and performance using the NIH toolbox
developed at Northwestern
plus a number of other tasks
that emphasize everything from cognition
to personality to sensory processing,
intelligence and variations on those themes
and the ideas did not only acquire
extensive phenotypic data
to the extent that's feasible acquired
under an umbrella that will be similar
to other projects that are emerging
that are exploring behavior and connectivity arrangements
magnetoencephalography is a powerful technique
in terms of giving access to fine temporal resolution
literally at the millisecond level
but it's very limited in spatial resolution
but especially since we'll be acquiring it from twins
who have also been scanned with fMRI
we expect it'll be very interesting
to explore resting state networks
and functionally activated networks
in individuals using this complementary modality.
I spent a couple minutes talking about brain parcellations
we're gonna really whiz through this
I alluded to the problem 30 gigabytes per dense connectome
makes it tough to analyze and parse out
the essential network relationships
we want to be able to strip this down
to what we call a parcellated connectome
where we think the neurobiology supports
on the order of 500 by 500 distinct parcels
but we don't have the parcels in hand yet
we're getting progressively better approximations
to that through fMRI derived analyses
but that's still a work in progress
and over the course of the connectome project
we'll see progressively better parcellations
but also challenges in managing the evolving domain
of how to subdivide the cortex.
There's also new methods that have emerged
Mylan maps is one that Matt Blaster in my lab
developed and is quite informative
we can take the ratio of T1 over T2 images
and that turns out to give not just colorful
but highly informative maps of Mylan content
in regions that fit with classical architectonics
and can be looked at in population averages
as well as individual subjects
and correlates with known parcellations
so we have confidence that it's well grounded
we can relate resting state functional connectivity
in an individual subject to a Mylan map
in an individual subject
and see that there are indeed strong correlations
between these two giving us confidence
that a multimodal approach is going to be very informative
likewise we can compare task evoked motion activation
with Mylan maps and show that both show hotspots
in the region of area MT that's been known
from other studies
and we can also look at this at the individual subject level
and see that Mylan maps and task fMRI activations
co-vary in their location high in one individual
low in another individual
and that sets the stage for using one or more
of these modalities to improve intersubject registration
which is high on our priority list
and we have some encouraging recent progress on that.
So lastly a reminder that we're going to make
all these data available in due course
and try to support a rich data mining platform
and portfolio for investigators here and elsewhere
to dive in and make some interesting discoveries
relating to brain behavior and correlations
of many sorts, heritability analyses,
genomic analyses and develop eventually
and in a nutshell to wrap it up
our objective is to understand human brain circuitry
and health and disease.
We'll get a huge amount of valuable data from this study
but we're not going to get a complete connectome
nor a perfectly accurate connectome
given the limitations of the data
so we have to be optimistic but maintain a critical eye
as we look at all these data
and hope that this will inform our understanding
of diseases development and aging in the long run
and so I will thank the members of my lab
who contributed a lot to the things I focused on here
and especially thank the entire women connectome
which has been a great team of people
to work with over the past couple of years
so thanks for your attention.
Okay, a little time for questions if there are questions.
I counted about six hours of participation per subject.
Sorry, the what?
I counted about six hours of participation per subject.
More than that.
So subject will come for two days,
they'll have five hours of scan time
plus all the behavior and everything else
that goes on at time in a mock scanner
so it'll be a full two day experience
and we have confidence because Andrew Heath
has worked with this subject pool before
that we will get good participation
but that's a critical part of actually making it work.
With the individual variability in your parcellations,
how do you imagine first attacking that
once you get the connectomes
and then adding that individual variability?
What software integration have you created
in order to incorporate for that?
It is a work in progress.
One is we've actually made some recent progress
with our Oxford colleagues in using the Myelin maps
to better align individual subjects
so that the task and the functional connectivity
and the structural connectivity will be better aligned.
So that's one part of it
but ultimately we'd like to say
the functional connectivity maps
should be used to further improve the alignment.
Conceptually, it's an elegant
and powerful way to go forward.
Algorithmically, it'll be a challenge
and we won't get that done right away
and we won't have the data to do the maximal optimization
so that's part of why we'll be seeing
an evolving set of atlases
that use more and more data to register
and parcellations that get better and better
and so it'll be something the field has to get used to
to say this is not a static picture of the brain
if we want to get the best resolution
that informs our analyses
and that's just part of the excitement
but the challenge on our side
and others who generate the data
and in the user community who tunes into it
and understands the richness
but also the limitations of what everyone is dealing with.
I'm wondering if you're thinking now
about using this data as a starting point
to address some of the issues that go beyond health
such as aging or perhaps pathology
and to overlay on the data you're collecting
biochemical data from MRS or PET or what have you.
We've talked about that aspirationally.
We were limited by practicalicalities
of what we could put into this grant.
There was enthusiasm about getting PET data
but we couldn't do that and everything else
but these particular subjects
are in principle accessible in future projects
to come back for follow-up longitudinal scans
looking at other modalities
as well as just setting the stage for separate projects
that can build on this kind of infrastructure
but explore other issues with other populations
but using the same kinds of informatics platform
if it benefits and makes life both more efficient
for working with data sets
and also sharing data sets across databases
which is something we're also very interested in.
I think that will, as part of how data mining
will expand in the future
but there are some devil in the detail
implementation issues.
Thank you.
So just a follow-up on that.
You had a slide earlier where you indicated
NIH Blueprint is to make use of these data
to ask some of those questions
but it occurs to me listening to what it takes
to get the kind of data you want
that in order for someone else to,
the bar is gonna be really, really high
for someone to attack some of those problems,
aging and the like, if they wanna do,
they wanna compare them to your data.
So there's an abstract version of the findings you'll get
and then there's the detailed version.
How do you see that playing out for other groups
who wanna get in the business of using your data
to ask questions about mental health?
A great question.
There are already large-scale projects.
There's one at New York University, NKI Rockland
are acquiring the best data they can
that are similar but not identical to our methodologies
and others are gonna be welcome to use as much
or as little as they like of our protocols
and our infrastructure for handling it.
So the bar will be lower by freely sharing the tools
as well as the data.
It's still gonna be a challenge
and we're, for instance, very interested
in analyzing the NKI Rockland data set
as an independent cohort
that will be also made freely available
and will allow the field to say for two different data sets
when you do comparisons across groups
and that group will include different ages,
ranges and different mental health states.
It'll provide a basis for saying
how do we as a field make comparisons
that are well-informed and robust
and are not caught up by the pitfalls
of cross-study comparisons?
We don't have all the answers
but I think that the community can start thinking seriously
about how to solve that given the combination
of an expanding set of very rich, large data sets
plus the importance of customized, targeted data sets
that will still be very valuable in their own right
but I think the value will be added
if they're put into the context of these larger data sets.
But there will be challenges in doing that well
and investigators and study sections
will have to think hard about how to do that.
So, thanks so much for a great presentation.
Thank you.

