Evolving Complex Systems in Biology and Medicine
Sep 5, 2023
Feb. 21, 2013. BioEngineering Seminar Series. University of Illinois Urbana-Champaign
"Evolving Complex Systems in Biology and Medicine"
Dr. Hava Siegelmann, Director of BINDS lab and professor, Computer Science and Neuroscience & Behavior, UMass Amherst
So, welcome, Ava.
Thank you very much.
It's my second day here and every moment was a pleasure.
Either good people or good food or bad weather, but everything was great.
So great hotel, so send more people there.
So the talk is on evolving complex systems and I'll start with introducing the problem.
So what I'm interested in is to try to understand something about computation in nature.
So typically when we talk about computation, we think about input and some processes that
are inside the computer and output or behavior, which is a natural thing to think about organisms
in all kinds of scales.
So we can think about cells, that they have their own senses and internal states and external
behavior, the signaling that they go to, the movements.
So you can think of computation in nature and all kinds of scales from organisms to
the smallest piece that you can.
And the idea that my main interest has been in the last few years is both to try to understand
how biological system works and in this also to understand how we work as a complex system
in terms of our memory, our ability to learn, and also health issues.
When are we healthy?
When do we have some disorders?
And another part of my lab is trying to replicate inspirations for better technology.
And here I've been working in the last few years on human robotic interfaces and smart
AI.
So closer to here.
Yeah.
Oh yeah.
Is there a microphone?
I didn't warn you.
Okay.
I'll do my best.
Okay.
So many times when people talk about computational modeling, they think about statistics to take
the data that we have and to try to understand it.
Now we know that in biology many times where we have a lot of data, but we're not really
sure what to do with that.
So even wonderful data like one that comes from the human project, there are some very
clear outcomes, but still a lot that we don't know what to do with that.
And some questions that come is how we combine data from the genetic findings through proteins,
processes, and all other things that come up through the whole organism to try to understand
health.
And there are various ways one can think about modeling.
So we can try to focus on data-driven models.
In that sense, we have to be aware that we're missing lots of data.
So we get what we get from whatever measurements were given to us, and many levels, many scales
are unknown.
We can think about fine concepts or do data mining, which is something that I do a lot.
Here there is also some hardship because the system that we try to understand and to understand
the rationales and motifs that they show may evolve.
So as they evolve, they may have different characteristics.
Another thing is that when we think about system biology in cells, we need also to think
about the multi-scale view.
So it wouldn't be right to think about just the cells and their communication or just
one level in the communication if we try to understand the whole system.
So I'll tell you my view and some of work that I've been doing recently about it.
And the content has five parts.
And we'll start with the thought of why we're going to dynamical systems to try to understand
questions in biology.
So there's a few people here in the crowd that are really experts in dynamical systems.
So with humbleness, I will give you my view of dynamical systems.
It's a theory that consists of equations that describe to us how some variables change over
time.
And in dynamical systems, they give rise to all kinds of subfields that some of us are
familiar with from chaos theory to network theory and to complex systems.
The interesting things about dynamical systems that perhaps make them a good candidate to
think about understanding biology is that what I would call the hypothesis for dynamical
behavior is most related to functionality.
So many times we can have the animals with the same gene constructs.
And one of them is sick and one of them is not.
Or animals that seem to have the opposite direction.
So the gene is not the only reason to decide whether we are sick or healthy.
And what maybe is a decider is how the dynamical behavior within the system works.
So if we would say the changes in the system, like mutated genes or invasion of irregular
cells or even neuronal death, they harm the system only at the time that the dynamics
inside start being different.
So we lose neurons a lot and we're still not amnesic until we start getting more and more.
So we can still carry lots of changes and if there is no clear dynamical change, maybe
we don't have the disorder.
On the other hand, if you use this approach, we can start defining dynamical treatments,
which will mean that if there is a particular dynamical behavior which we are interested
in, dynamical treatment can be a mathematical way to try to restore the dynamics back.
And this does not necessarily have to be by treating the mutation.
Maybe it's impossible.
But maybe by changing the dynamic that it will go back by different means.
Maybe change signaling to add something to the system.
This is an optimistic view for clinicians that maybe there are ways back even if we
cannot fix the cause.
So I would like to show one short example for the dynamical systems in medicine.
And this example comes from the circadian rhythm.
And circadian rhythm is a great system to study because it's kind of something that
we all have and it really prepares us to the rotation of Earth.
So the system has to be such that we predict that night will come and day will come and
it will predict what different parts of our body have to do in different timing.
And the circadian mechanism is really multi-scale.
So you have within the cell, you have negative feedback of proteins upon transcription of
the genes that encode them.
You have the super charismatic nuclear, which is the master clock in the hypothalamus.
And it's really master clock in the sense that even if you have a knockout for the genes
that go for the clock, if you take a master clock from another animal, you restore the
rhythm.
And you can actually change rhythm by the master clock.
So we have the cells, we have the master clock, and then we have also an orchestra of the
oscillators between the different organs.
So every organ pretty much is synchronous to itself and there's some discussion between
them.
And that's how things work together.
The questions that come here, which I'll try to answer with dynamical system, is why the
body falls into jet lag.
And the reason that this is a question is because until 2005 or 2006, the assumption
was that the light responsive part of the SCN updates very fast.
And then the question, if it updates very fast, why the rest of the body doesn't update
very fast?
And the other hypothesis that was that the SCN moves with a change of light as you fly,
and the rest of the body, the rest of the peripherals kind of go after that.
And they just go in different rhythm.
And that's why you have some imbalance.
But this didn't explain any of the data.
Some of the data would show that your liver can think that it's day, while your skin can
think it's middle of the night.
So there are kind of things that were not explained very well.
And the idea was to try to answer with dynamical system view.
And here, the particular interesting thing is that up till today, there are many unknowns
in the circadian system.
So for example, we don't know exactly how the master talks to other parts, to many other
parts of the body.
And we don't know how various parts of the body synchronize between themselves.
So we have to go to mathematics to fill up the missing parts.
So the only thing we would like from the model are just four simple properties.
And these are the four properties.
One property is called free running.
And free running is just if you put the animals in total dark condition, you want the circadian
part, the cells, to tell you it's day, it's night, it's day, it's night.
So to keep the rhythm, even if there are no signs from the outside.
The other property that we want is what is called the phase response curve.
And this is a particular property of an oscillator, that if it's all the time in dark time and
you put light, what changes will happen to the oscillators?
And the changes that are seen many times is that the beginning of the night, there will
be a delay.
And at the end of the night, there will be an advance.
So this is a property of all circadian oscillators within cells.
So that's another property that we would like.
The property of entrainment is just the ability to entrain to light and dark cycle.
So if you put the animal only in dark, it will have its own day and night.
But if you move it to a particular country, after some time, it will get used to the 12
and 12, or the 11 and the 13, or winter and summer.
And the reentrainment, which is the last property that was studied a little bit less at the
time and now more, is what's happened when you actually move from one area to another
area.
What's happened in between?
And this ability to reentrain.
So you already were trained to particular day and night.
And now you're in different days and nights, in different times, how you reentrain to them.
So these four properties we want to have in a model.
And the idea of the dynamical system modeling is very interesting.
So what we suggest to do is to take a particular equation.
And to say this equation has this oscillator, is this oscillator that has these four properties.
And interestingly, it is used also for describing protein and mRNA in cell.
But we're going to describe peripherals, different tissue that behave like that.
And I will try to put them together.
So this particular oscillator system is really a feedback loop.
So we can kind of see that the M and the P create a loop.
So this is a loop thing.
And the way we couple them together is just by two properties.
One property is how if the peripherals are self-sustaining or not.
And the way you know it is you take a tissue from liver and you put it in a dish and you
see how long it takes until it loses the oscillation.
So that's how much is self-sustaining.
And the other one is this kind of knowledge which tells you what's the phase lag.
So if this is the SCN, some seems to go before and some seems to delay after.
Oops, that's me.
Let me close it.
Sorry.
Did I close it?
I think.
Okay.
There are a few parameters to fit so that you get the right dynamics for reentrainment.
And the results that we get are kind of intriguing.
So what we get that even the first part of the SCN that we thought is getting tuned in
one day, if we go to faraway days, if we go to six, seven, eight days, sometimes we don't
get exactly in one day to where we need, but we can have this property of overshoot.
And if the first part of the oscillator overshoot, the second oscillator that stand behind it
will have different kinds of behavior which can delay and then advance.
So now when you bring it all together, you find an interesting property that when you
have oscillators, according to this dynamical system, which are combined in a very simplistic
way, really kind of data-driven of phase lag and basic things like this, you find out that
when you move eastward, when you advance your clock, what's happen is that the ventrolateral
part, the first part that is light-sensitive overshoot, the second part start delaying
and then going forward, and the other peripherals are not chaotic in the sense that they just
do something.
They are very clear what they do.
Some will go directly after, and some will go all the way around.
And now the data is explained.
So at some point, some peripherals will be here while other will be here.
There is a good ending.
The system is flexible, and they meet at the end.
So with this kind of modeling, you can come in and start thinking about what kind of protocols
of day and night to do in order to avoid these kind of movements.
And we suggested this particular one, which was checked and validated for rats in Mary
Harrington lab.
So this is for rats, but Forbes were kind enough to continue the research by themselves
to say that it's also true for people and even give me the credit for that.
But I have not checked that myself.
But I would love to.
So it will just mean that we'll have to send all of us around the world and ask us to urinate
in small cups and checks for chlorine and other things.
So this is definitely something that we should do.
So this is an example for dynamical system modeling.
And I would like to go to the next step, which is the complex system modeling, which start
getting us to something even more interesting.
So complex system bring us to take the idea of dynamical systems and takes us into something
that is more practical.
Because here we have units which communicate between themselves.
There is interaction.
There is more clarity of the multiscale.
And if you want to interfere, it will not be any more general mathematical connection,
but will be something that is more related to the system itself.
So for complex systems, we can think that it's units that are separated, but each of
them is dynamical system that does local rules.
But together they emerge memory and behavior.
And we can think of population dynamics, because now we can have units from different type
of populations and focus, I think, mainly on interactions.
There are various examples that we can see actually everywhere.
And the three that I chose just to demonstrate are neuronal death, post-stroke, there is
malfunctioning of the cell signaling, which seems to be the cause for the neuronal death.
If we talk about tumor growth, it can be also that the healthy tissue around it lost some
of the rescue protocol, so it wouldn't be only the fault or the tricks of the tumor
that plays, but other players also started doing other things.
Or bacteria, that there are all these beautiful examples of how they possess a joint memory
after some experience.
So all these are examples of complex systems that through interactions start behaving as
one thing and give rise to behavior.
Here are the particular examples that I will give here is research in cancer.
And there are many types of tumors and cancers, and they are different by their origins and
by the gene changes, mutations.
And all this partially predict the aggressiveness.
So for various cancer, people say, you know, what's the five year limit?
But it's not one to one.
And a lot of approaches to tumor growth is to look at different tumors and try to understand
them, which is the right way to do medically.
But since I come from different direction, which is more like the computational one,
I'm actually curious to ask what's common to solid tumors in the tissue level.
The tissue level is interesting because it kind of combined things that were inside the
cells, and it talks about the signaling between the cells.
And maybe there are some predictions that one can try to come with.
So the objectives here are to find what kind of properties which allow previously healthy
tissue to allow tumor that maybe wanted for years to start to actually be able to start
and take over the system.
And the hypothesis is that there are conceptual interactions among cells that could guard
against tumor.
And they worked for many years until something happened.
And perhaps kind of simulations of protocols that can allow us to do that.
Now, as you will see, I have different types of simulations.
So the way that the simulation look, I kind of took one piece of a square just for in
order to show it here.
And here we have numbers of stem cells and total number of cells.
So I just separated two kinds of cells.
And I do it for simplicity here, just in order not to go to epithelial or to particular type
of tumor.
And on the right side, we kind of have a very simplistic way of the geography just for us
to be easy to see.
So all the colorful lines are healthy cells, and the black will be the tumor.
So when we talk about interaction between cells, we can really think about two kinds
of protocols that cells are doing.
There are those that are life protocols, and there are those that are interaction protocols.
And you can really think of any kind of protocol that you have, any type of communication,
as either being life related to proliferation or something like that, and something about
signaling and interaction.
So the interesting part for us will be more about the interaction.
For the cell life protocol, what really happens is that you can start with small stem cells,
and you can grow because of all kinds of properties.
And what you'll have is some status quo.
So the life protocols will be good to have a good, healthy system.
But the rescue protocols or the interaction protocols are those that will be important
when you can have mutations that will lead to cancer.
So in that case, when you talk about rescue or healing processes, what we would like to
do computationally is to look for such interactions, without thinking of the biology for a moment,
saying, if I were to design a system that has different cells with some particular properties,
I would like to have these properties.
I would like to have cancer eradication to avoid the healthy system death.
And I would like to have long-term stability in the sense that after a fight that won,
the system will be in a situation to be more alert.
So we would like to have some kind of ability to simulate cancer growth, and then some ability
to think what we can do about it computationally again.
The idea here in this kind of work is really to use the computer to do for us the lab work.
So we would like to think of numerous hypothetical protocols among cells.
So when we talk about protocols, there are those cells that initiated the signals.
There's the causes that these signals initiated.
There are properties of the signals, like the strength and the radius, how long they
stay there.
And then the receiver action, what happened when you received the signals.
So these are protocols, classical in computer science.
We try to apply this idea of protocols to here.
And here the idea was to study it under different conditions.
Population heterogeneity is a must in this field, because otherwise one thing will kill
the whole system.
There is nothing to talk about.
And then other things like relative proliferation ratio and different properties.
And if we just run it and looking for the most simplistic answer, we can see that ideally
in order to have simulation of growth in a healthy tissue, we need two types of protocols,
ones in which healthy cells start, which we will call please die, and ones that come from
cancer cells, which we'll be calling here I'm dying.
And this combination, this joint citizenship, if we had it in a tissue, we would have very
good ending.
So what are these?
The please dies are the ones that the healthy cells, cells when they feel obstructed, and
the receiving cells can do different things.
They can either initiate apoptosis if there is a lot of please die, or if there is a little
bit of please die that they know something is wrong in the area and they have to increase
their alertness.
And we know that there are some different types of protocols that can be described as
please die, both from the immune system and from healthy surrounding systems.
The other type of I'm dying are a little bit harder to understand.
And here we think about cells that die due to protocols.
And these are both healthy and cancers.
And when they say I'm dying, it goes smaller areas.
But what is nice about it, if it's high threshold, it's supposed to start apoptosis to the neighbor.
If it's low threshold, it should elevate proliferation.
And this is for the healthy cells.
So again, the I'm dying is a partially theoretical protocol, but there are some correlations.
So we know that there is some knowledge about cells that enable the rest to know that they
are dying for various reasons.
And if these two work together, then we can get some signaling that we will see in yellow
in which the cells around the tumor will try to fight back.
So there will be the signaling of please die, but I'm dying.
If you go for only one type of them, it's not working.
So if you have only please die, the tumor pushes.
So in the peripheries, there will be a little bit stopping, but not much.
The I'm dying cannot start by itself.
It has to start from something that came from outside.
So it's really a way that you need properties from outside and properties from the inside
computationally or mathematically in order to start there.
Now, there is an addition of the blood vessels, which is an important part in the tumor.
And here, as you see, the blood vessels that we put are very much in a matrix form,
because it's kind of a classical computational simulation.
As you see here, we took the cells at the end out, and you can see the shape of the
angiogenesis that was formed.
And the same thing, if we allow the protocols to work, so we have these blood vessels that
already come to the tumor to give nutrients and to hold it and to do everything that is
needed to stop it.
And even with these two types of protocols, the fight is not easy.
So we can miss it, and it can go to another part.
I do not show here different parts of the body, but they can still go to another part
and make it even harder to control.
The interesting part, I think, in the protocol is that even if it sometimes doesn't work,
the system still works.
And here we show that if there's 60 percent of the system is taken, it can still work.
Or if a large amount of the cells do not participate, even if only 30 percent participate,
we already have high percent of success.
So for this part, what we can think is about a complex system approach of population dynamic.
So we have the cancer cells here, which are one cells, and we have the healthy cells,
which are different cells.
In reality, we're looking at different kinds of cells which are related to this part of
the body.
But there should be some messaging, not only on population, but in between, and different
behaviors in order to do more of that.
So we saw something about dynamical system modeling, and then about complex system in
which we look at units, and about the communication between them.
The third part, which may start making it a little harder, is the fact that we are talking
about biology, and in biology, the system evolves.
And we all know that currently there is no real good treatment to most cancers.
And the reason is that the treatment works up to some time, and then the cancer cells
have changed, and the treatment no longer works on that.
Or we know in viruses, or in other cases, that there was one thing, and now the system
at hand is something else.
And because of that, there is a hardship to characterize the system.
But this is actually our goal, to come with a way that we can describe the system, and
describe systems that can change over time.
This is a very hard type of work.
And I will show you just very few examples on that.
So one example, which was done with these colleagues, was in the field of addiction.
And here we try to understand how the miR-9, which is the micro RNA, just have, if you
put a tissue in an alcohol, after 15 minutes, there were some changes, small changes in
miR-9, that causes such changes in the neurons, which lead very fast to high percentage of
alcohol tolerance.
So the system has evolved just from putting alcohol near.
So this is like one type of process.
Another process, which is much high level, is the process of reconsolidation.
So this is a process that we all live through all the time.
And the idea in reconsolidation is a process which is different than the one time consolidation.
So first time you see something, so you consolidate this in your memory, and that's it.
But then when you don't look at it, and you try to remember it, you bring it up to your
memory, now there needs to be another process that put the same thing back to your memory,
or else you lose it.
So this is a process that is known since 2000.
And since then, there was much work that was done in trying to use the process of reconsolidation
and disrupt it for memories that are not wanted, like in PTSD or in fears.
So in the last few years, if you open ACER in science, you will have a lot of how to
disrupt reconsolidation so that memories that are there will not go back.
My interest was more in why do we need this reconsolidation?
Why do we want to have this kind of memory that changes all the time?
And is there a way that we can explain it?
And we're working on some modeling of reconsolidation that try to explain how human
update the memory all the time.
Try to put it in a models of a tracking, and also in models of robotics.
But these models are particularly tricky, because what happens is you start with one
system, and you have to allow it evolve with input and change.
So every time you have a different kind of system at hand.
So at every moment of time, you have a system.
But in two different points, the system is a bit different.
And there is a third example that maybe for time, I will just say very fast.
There is this wonderful information that we have now from the last two years about
parts in the brain that are organized into small clusters of about 30 neurons that are
clustered strongly together.
In the cluster, there is almost no learning.
And in between cluster, there are all kinds of interesting type of learning.
And we try to model and to characterize what you do and how these things evolve.
So what we saw so far is something about the view of dynamical system going to complex
system, which is more practical and kind of allow us to try to make changes.
Mentioning that this type of dynamical system that we model the cells should be such that
we can change them over time, or else we're stuck with models that are not biological.
Are not biological.
Are just kind of classical and not changing.
And here I want to talk a little bit about how to work on computational modeling about
such systems.
How much time I have?
OK.
So I'm doing great.
OK.
So all the models of computation, all the models of any computer that we use now, any
type, you know, your laptop, the supercomputer that is here, watches, everything, insulin
pumps, is all based on this great invention from 1936 by Turing.
And his genius idea was that the idea of computation is that you have input and output.
But instead of having a controller, that the controller itself is the hardware and the
program, he allows to have two different things, loadable programs.
So you can load whatever program, program for summation, program for multiplication,
everything to the same medium.
And you can have memory on the side.
So if you have your own controller, that's what people had, you know, until the 40s.
Now you can add loadable program and memory.
And this gives rise to the general view of computer, which is something that has I.O.,
memory, and the CPU, where the program is being done.
Now this model from 36 has been so basic and so used that it really was, has been used
in all computer designs, in computational explanation of nature, and in A.I. and technology.
This year was the 100th year for Turing.
So some of us were in conferences and other of us may have heard television or radio program
that told us all kinds of things about Turing.
So currently, he's really considered a genius father of computer science.
So in 36, his 36 model is an undergrad project.
So I tell this to my undergrad, that's the level I expect them.
And two years after, when he finishes his PhD, he's already coming with the notion of
subroutines.
There is no computer, but he already knows what programs are and what subroutines are.
Immediately after his PhD, he goes to break the code of the Germans.
He's invited to the National Physical Lab to build the first computer that will simulate
this will be universal computer.
Three years after he writes the first manifesto in artificial intelligence, when there are
nothing yet, he's already talking about neural networks and about random changes.
And then he's moving there and he's leading the projects for the first computer which
is using industrial.
And all these things are known about Turing.
There are some things that when you look at history and nature and science papers that
wrote about him in the last year, there are some parts that they didn't tell us about.
And I think actually this is a very important part of his work.
Maybe something that in years to come, we will consider his most important work.
So I want to say first, what's the problem with the Turing model of 36?
The problem is that based on this 36 model, people will consider the genetic material
to be the program, the proteins to be memory.
And this is almost true, but not really.
Based on that, neuroscientists were looking about discrete models of the nervous system
on off and on memory as being very stable.
But we know that reconsolidation and our dynamic memory is not this fixed reliable memory.
So it's not exactly the truth.
So the first person actually to notice that the Turing notion of computer is not sufficient
to describe nature was Turing.
So Turing tells us in 1950 in a paper that his idea is for electronic computers are for
them to work like those human calculators in a disciplined but unintelligent manner.
So he's not expecting intelligence from his computers, but he believes that there will
be those machines that will be really intelligent.
And interestingly, already when he's trying to build the first computer, the first universal
computer, he's going away from the original plan and try to put other things.
And his boss at the time, that also is called Charles Darwin from the same family, is very
upset with Turing.
He said that Turing is trying to put higher facilities to this computer.
And this would be a theoretical work better be done away from there.
And Turing leaves to a leave of absence and never come back.
So that kind of almost put the end of that kind of research.
But Turing is not a person that will stop this work, even if he has a different day
job.
His night job continue actually all his years to look for them.
And to look how he thinks to have different kind of model is really to look in his writing
and in his interviews and try to see.
And he has different ideas.
He has ideas that he already suggested in his PhD thesis.
And he talks about this series of machines.
So instead of talking about one machine, we have a series of machine.
Each of them will be better and better for what we need.
So think about heterogeneity in nature.
OK, we don't talk about one cells.
We have the virus and the body, and it's all the time better.
It's not perfect, but it's going through the next flu.
Then he talks about learning, which will allow us to go through that.
Then he talks about randomness.
And he really believes in randomness for free will.
And he was talking seriously about supply of radium.
And that's what he wanted to add to the computer that he built, which made him lose his job.
Then he's talking about rich information.
And he says, you know, the whole idea is that the machine will have information that you
don't have.
And it doesn't matter if it comes from randomness or just that you don't know what's there.
And he talks also about analog computers.
He said the brain has to be analog.
And discrete is a great idea for many things.
But if you want to be more like a brain, we have to think differently.
Unfortunately, he died in 54.
And some years after, without being aware of this kind of work, I came as a theoretician,
a computer science theoretician.
And I ask a totally different question, which I ask, we have models of neural networks.
Can I consider them to be a computer?
And it's not clear that you can.
It was not clear to me.
Because a real computer, like a Turing machine, you have memory and you have program.
And when you talk about neural networks, what do you have?
You have weights, which I don't know if they are the memory or the program.
I can't really separate these two.
And every time when I update something, I also updated the program.
So I'm not really sure what program I'm running.
So the first question came, is it a computer?
And we were looking at analog recurrent neural network.
Analog means not to discretize anymore.
No reason to discretize them.
General infrastructure.
And the finding at the time in 93 was that, yes, we can simulate a universal Turing machine.
So we can consider recurrent neural network to be a computer.
But then I was very upset.
And I'll share with you why.
As a theoretician, when you have something that you prove is strong,
you have to also show that it's less than Turing machine.
Because we were told up to some time that everything is bounded by a Turing machine.
That's how I learned it.
And later, I learned that Turing actually said the opposite.
But that's how I learned it at the time.
So I tried to prove that it's less than Turing machine, and I couldn't prove it.
So I thought to go home, not to tell my advisor that I'm such a failure.
I went home, and I said, OK, I'll prove for myself at home that these neural networks
are more than Turing machine.
And as I try to prove it, I will fail.
And I will see what's the weak points.
And then I'll take this weak point, and I'll turn it back.
And I'll show that where they're bounded.
So I tried to do that.
And I proved that they're stronger than universal Turing machine.
So I wasn't really sure what to do.
But mathematically, it was sound.
And we checked it.
And we published it.
And we were not really sure what to do with that.
And then I say, I want to characterize what it means to have super Turing computation.
And I started seeing that it has particular properties.
We actually have a whole hierarchy between Turing and super Turing.
So it's not just two things.
You have a whole hierarchy based on information that you can use in the system.
You can achieve it by things like randomness.
You can achieve it by learning.
It actually gives you something that is very confusing.
It's actually, you can think of super Turing as a series of machine
in which at every stage, when you look at the machine, you have the Turing machine.
But in two different times, you have different Turing machines.
So I say, if I have a machine, and I start running it in my lab, and I stop it, I say,
OK, I look what happened from the beginning to now.
I can describe it as a Turing machine.
And that's the end.
Now I let it run another day.
And I'm looking what I have now.
I say, oh, that's also a Turing machine, but a different one.
So we get kind of a series of machines, which is a really strange idea.
And the notion of analog comes here, too.
And it was not until two years ago that I got into this great book
of Jack Copland, who is a philosopher.
And I kind of went back to see that it's very, very strange
that we came from such totally different questions, what neural network or complex
system are doing, or what is intelligence, which are totally different questions,
and came to the same principles.
So Eduardo Sontag has been my advisor and very, very active in the whole thing.
And the idea was to show that neural network are really versatile model of computers.
You can have Turing.
You can have super Turing.
You can have all things between.
And you can have all kinds of interesting properties.
What I would like to mention now in the last five minutes is that people justifiably came
with very interesting questions to ask about this kind of strange idea of super Turing model.
So this is a strange idea.
And I agree.
So I would like to mention a few questions that I think are very valid and are very interesting.
So some people say, well, for the neural networks, you use infinite precision in their weights.
You use real number in the weights.
But the truth is that while we say that there are real values in the weights,
we actually never use them.
So we have this lemma that says linear precision suffices, which means if you want to run,
if you discretize time and you run 20 steps, you need only 20-bit precision
in the weights and in the neurons.
So physicists know it.
They have chaotic systems.
And they know that the systems will go away with linear, with exponential divergence.
And exponential divergence in terms of bits is linear many bits.
And you have exactly the same properties.
So except for there is some divergence.
So we need linear precision in the bits in order to make sure where we are.
So we can actually cut.
So we don't use linear infinite precision.
And Turing explained it so much better than I could.
In a paper that he wrote, he said there is no theoretical difficulty to have unlimited
storage as long as at every time you use finite part of it.
Another question that comes is what's happened with noise?
So in computer science and probably in classical engineering,
many of us are very worried about noise if you want things to be bits and clean.
And what do we do with the noise?
If we really want to have noise, we can go to evolving networks
with a little bit of precision.
So we don't need long precision.
So we can have a little precision but evolving.
Or maybe something that I think about it sometimes.
Maybe we like this noise.
Maybe we can do something good with that.
I don't know.
Another question that Martin Davis, a mathematician,
brought, and has been brought many times,
which said we kind of play a game of non-computable real numbers in these weights
and in these networks.
And the world doesn't have non-computable numbers.
So my answer is that I don't know.
According to Turing, the definition of non-computable numbers
are those numbers created by repetitive, non-intelligent steps.
I'm not sure that's how the world is built.
Or if we take random bits from around us or from different processes,
their number will be non-computable because they're not coming from one organized bit.
So I'm not sure this is a good question.
But a very good question that I've been asked many times is,
are all hypercomputers realizable?
And here I would like to note a difference.
So super Turing is a term, is this kind of network that I started showing.
So it's not only this network.
It becomes this family with these features.
But this is something we suggested with the Duardo E93.
And it's based on Turing machine with some updates to that.
Hypercomputation is a term created by philosophers,
Copland and Profund.
And they put inside everything which is non-Turing.
For example, they have a machine that every step runs twice as fast.
So they don't have any problem about the speed of light.
They continue to run faster.
So we don't have this kind of machine.
So I think in general, hypercomputers are not realizable.
I suspect the super Turing is realizable.
And there is another thing that I hear sometimes.
I was in Berkeley some time ago.
And I gave a talk on super Turing.
But the real talk with all the mathematical detail.
And at the end, some very famous person stood and he say,
I'm a soldier in Turing's army.
And I was thinking to myself, oh no.
Turing was very strong about finding things that are beyond the Turing limit.
He's lost his job for that.
I don't think he wants this kind of followers.
But I think the best answers to computing that is evolving type of computation,
which is what Turing is talking about, comes from Turing himself.
So in 48, he write in his paper where he introduced neural networks.
He say that he thinks that many people will be against it.
And it's still before there are real computers.
No computer science department.
He already know computer scientists better than other people.
And he write nine objections that he thinks people will bring.
And he answered to some of them.
So I brought just the ones that I particularly found interesting.
One is what he called the Geddel objection.
Math is incomplete for all kinds of things.
But he say it's true math is incomplete.
But it's not true that people are complete.
And if we were not allowing people to make mistakes, we would be stuck there too.
So he say it's unfair to require math and computers to be perfect.
Then the theological thing that only human can thinks.
He say, you know, this is not a scientific argument.
It's a nice argument.
The head in the sand, he called.
For we cannot allow computers to be stronger than what we put into them.
Because it will be disaster.
He say that's nice.
But it's not an argument.
Or the Lady Lovelace objection, which he said that current machines
are only given what you gave to them.
And he say that's true.
But maybe there will be other machines.
And he finishes his 50 paper with this looking for the future.
So the whole idea of super-turing computation is that
either it's a method to build computers which are intelligent.
Or it's a method to try to think about biological processes which are evolving.
They have the same, very interestingly, the same principles.
And I'm not saying it by any pride.
I'm saying it with full humbleness.
But it's very interesting that you come from different direction.
And you kind of come to similar ideas.
And the question that comes, should we try to realize it?
So Turing said, oh yes.
And he has general thoughts about how to do the learning.
But I think a little bit differently.
Because I don't come from the idea of understanding intelligence.
Because I don't really understand intelligence.
But I think that if we can see us here, how biological systems evolve.
We come into processes that allow this changing.
If we can put this thing into the machine, maybe that's what we get.
Maybe that's the new machine that we want.
So I want to finish with kind of future directions.
So in 2011, you know, the computer science community was very excited.
We saw this IBM Watson winning Jeopardy games against a very impressive group of people.
And this was a high point, kind of the ceiling or the top
of what we can do with Turing computation.
With huge clusters of databases, with clever programming.
So in one way, this is like the best we could have ever wanted.
If we think biology, we say, well, this machine lacks flexibility.
Go to change the rules of the games.
And I don't know what will happen.
So for future technology, we really want to think about systems which are smart.
Which are either more autonomic.
And Turing said it very nicely.
If the machine is treated just as a domestic pet and is spoon fed with particular problems,
it will not be able to learn in varying ways in which human being learns.
So he talks about intelligence.
I think he really talks about biology in general.
So this is one project we're trying to build analog hardware with these great guys.
And with another project, we try to understand a little bit better computation and neuroscience.
And to transfer some ideas to interfaces and robotics.
And this is with Henry Markram.
And the hope is really to get wonderful simulations that can help us in medicine.
And this brings us to the other side.
Which is maybe we have from here some kind of understanding of the future of medicine.
So if we really think about trying to explain data that we have as complex systems which evolve.
Then maybe we can provide a dynamical health hypothesis.
Which says that just through the restoration of dynamic in different ways.
Can be beneficial to what we want to get.
Which may be just a different way of thinking about problems in the body and attaining health again.
So I just want to thank everybody for coming to the talk.
So I hope I didn't confuse you too much.
I was trying to give a talk which is not like one particular thing that I like to do.
Thing that I like to do.
But more like kind of an overview of going forward.
So I hope it was not kind of too much information in one hour.
You showed a number of really interesting applications of computational modeling in medical problems.
And I call jet lag a medical problem.
What kind of data do you really feed into the models that you're elaborating?
Where does the say ground truth come from?
Is this from collaborating people?
Yes.
So it's all from collaboration.
So the jet lag we collaborated with two labs.
One at UMass where I am.
And one in Smith College.
And the work that we're doing on the Lego block structure.
We collaborate with the group with the Blue Brain Project.
This is an APFL.
So you know whatever we can.
When coming and looking at this department.
This particular one.
There are so many imaging and so many signaling levels.
That I'm just kind of waiting to the next kind of model.
That will have kind of signalings in different multi-scale levels.
And we'll get something out of that.
So I don't have my own lab in terms of a lab.
Maybe on the presentation on modeling the behavior of complex mixtures of cancer cells.
So again in practical terms.
What kind of experimental setup was behind the data collection that leads to the modeling?
So the particular work that I was looking is part from a work that we look at breast cancer.
And we collaborate with two different groups.
One is clinicians.
That actually are talking about treatment.
So there is a part there.
That I kind of jumped very very fast not to get into that.
Which is trying to put treatments together.
And another one is in UMass in the veterinary sciences.
Is a group that works on mice breast cancer.
The beginning of the work was looking at different cancer.
That start either in stem like cells.
Or ending cells that go back part way to cancer.
And the original questions were to differentiate this kind of cancer.
Which is why we had counts of them.
So this comes from data there.
Currently for this particular work we're looking at five
functional families of breast cancer.
And try to see what's happened with the details of them.
And we have another group that is a visiting professor.
That came for a year from Korea.
That is now in my lab.
And we try to connect between the genotype and the phenotype.
So this kind of simulation is very phenotypical.
So everything was translated to behavior.
So there is nothing about particular factor or anything like that.
It's all called by names according to behavior.
How much they're sensitive to space.
How much they're movable.
How much they're listening to that kind of signaling.
So with her we're working at going back.
We kind of started by going to that direction.
And we start by going back to build this missing level.
To make it a little bit more down to.
So we started actually more clinically.
Because that's where we started the work.
And now we're going a little bit more down.
But every data.
So the fun thing about this computation and models.
Is that every data you can put in.
Is telling you another layer of understanding.
And the good part is if it doesn't work.
You run it again.
You don't have to do big things in lab.
Yes.
Please.
So for this particular model.
The actual prediction that we're working on.
Is that angiogenesis is helpful.
At some stages of the breast cancer for medication.
Is helpful in medication.
In terms of therapy.
Not only in terms of growing the cancer.
But at some stage after some behavior of the cancer.
You can actually use the angiogenesis for your own good.
So meaning that if at some stages.
Your choice of treatment is to ruin then the angiogenesis.
You actually allow the cancer grow instead of fighting it.
So a particular prediction is actually what you can do.
After reaching some network of the angiogenesis.
In order to use it for the benefit of fighting the cancer.
It's a little bit anti-intuitive.
Computationally it's very interesting.
And maybe there is some sense.
If you think about it.
If you put medication through these blood vessels.
And they are the ones that are diffused.
You can reach all kinds of places.
If you were creating them and then killing them.
And you had already some structure.
And the tumor was continuing or metastasized.
Because it didn't have cancer.
It didn't have enough food there.
It would maybe run faster to stronger blood vessels.
And go to another place who started going with metastasis.
While if you go at some stages.
It has to be some properties of the cancer at that time.
And at that time you kind of wait until some moment.
And then try to bring a lot of chemicals through these vessels.
Maybe the way that they will work will be like the signaling that we showed.
What type of breast cancer was this?
What medicine did you use?
So in this modeling it's kind of totally not tried.
It's a mathematical treatment.
Oh but there must be.
But surely the efficacy of the medicine relates to how fast it diffuses.
Or is otherwise distributed.
So we think of families of medicine.
And the timing depends on the type of efficacy.
So it's a family.
It's a family of chemotherapy.
So you talked about three protocols.
Dying.
Where the cells are dying.
And then the rescue.
So these three protocols.
Do they get distinguished in some way?
Using molecular data or any other type of experimental data that you input?
So the idea of looking at the cells as having different protocols.
It's my label of how I call these protocols.
So from the cell point of view.
They have protocols of behaving.
I call them life and rescue.
Because the rescue ones are the ones of interest to find.
So the life one I think they're pretty well known.
They're unrelated necessarily to cancer.
They're cellular.
You know life cleaning.
Fixing.
Repairing.
You know all kinds of things like that.
Which is kind of good for a good cohesive system.
And the rescue protocol were the focus here.
So I named them rescue protocols.
But it was just a combination of a few protocols that together can do something.
Now it is my belief that there are not just two things that work together.
It's very clear that there is a family that will do more.
This is kind of a minimal version of properties that you need in order to be able to run after.
In a fast way.
So there is definitely the work from going to here back to the biologists to the medicine.
Is to be done.
We're out of time.
So let's thank Dr. Susan.

