Week 8 Lec 16

IIT Kanpur July 201
20 followers
Mar 12, 2024
[Music]
resource allocation in private and
public Edge Cloud systems I am Dr rajim
Mishra working as professor in the
department of computer science and
engineering at Indian Institute of
Technology Patna so collaborative Ed
cloud computing here the edge notes
usually do not exhibit
enough storage and Computing resources
when processing the massive data
therefore a collaborative Edge cloud
computing provides the
users with app Computing Services and
requires the cloud and Edge
cooperation in satisfying the
requirements of computing and storage
resources now this collaborative Edge
cloud computing scenarios The Edge node
with limited local resources now can
rent more resources from the cloud node
and pay corresponding cost to meet the
users
demands now according to the nature of
the cloud service cloud service can be
divided into two different type of
services one is called private Cloud the
other is called public cloud services
now regarding the private
Cloud so when the users Computing
demands randomly reach the edge nodes
The Edge node need to decide how to
reasonably allocate resources between
the cloud and Edge nodes to satisfy the
users's demand now for Edge notes
exhibits its own virtual machines to
process users demand however because the
the number of VMS requested by the users
May exceed The Edge node capacity so the
edge node now needs to rent the virtual
machine from the private Cloud to scale
up its capacity now also the cost of
private Cloud changes dynamically
according to its physical Computing cost
so the edge node need to dynamically
allocate resources at each time slot
according to the resource allocation
policies now in public
Cloud the service provider that is from
the public cloud services offers various
pricing modes for cloud service so the
edge node also needs to select
appropriate pricing mode of cloud
service for collaborative Computing
according to the users demand now the
cloud CL service providers that is
public cloud service providers such as
Amazon
Microsoft provided three different
pricing modes Each of which exhibites a
different cost structure the first one
is called On Demand the second is called
spot third is called reserved so on
demand requires no commitment that is a
cost the most but offers uninterrupted
service that is called On Demand
the second type of service which cloud
provider provides is called
spot so it is the highest of saving yet
unpredictable and not for long-term
projects third type of service which
cloud provider type of service is called
reserve one or three years commitment in
exchange for Discount service now for on
demand instance for from the cloud
service provider as a service the this
pricing mode allows the user to pay with
a fixed time grity set by the platform
and the price is fixed in a long period
of time reserved instance in this
pricing mode requires the user to submit
the reservation time in
advance and pay the corresponding
reservation fees to have instance within
the contract time now it is applicable
to the users with a large number of
computing demands and unit price is us
usually approximately 50% lower than on
demand instances the reserved
instance has an option of customization
with additional cost associated with it
and called upfront cost third is called
spot instance the instance of this mode
is obtained by beating and its price
changes in different time slots which
usually used for short-term Computing
demands so because the Computing cost of
the edge not changes dynamically
according to their workload if no
strategic resource allocation exist when
the
edge when when the collaborative Edge
cloud computing provides the service the
Computing resources in the EDG note with
a relatively low Computing cost cannot
be used reasonably and its Computing
cost will increase therefore reducing
the long-term operation cost on the
premise of meeting the dynamic Computing
demands of the user is a key problem in
CC that is collaborative Edge cloud
computing the goal is to achieve the
lowest operational cost under different
strength of demanding amounts and
Computing time
durations so the following sections in
the next slides we will formally Define
the resource allocation problem in both
private and public Cloud
scenarios so resource modeling the
resource allocation problem in
collaborative Edge cloud computing now
let us assume that each time slot T the
user submits a demand for the resources
the user demand can be represented as DT
D that is the demand at time T is
splitted into DT and LT so DT represents
the number of virtual machines which are
being
requested and T represents the duration
of the service requests in the time
slots at a time T this is the
demand is the user
demand now the total Computing resources
that is VMS owned by The Edge node is
let us say represented by the E so the
resource is allocated to the user ET to
represent the number of remaining
virtual Machines of The Edge
node ET is the
remaining remaining virtual machines so
the number of virtual machines which are
provided by The Edge node is is
represented by DT of
e provided by The Edge notes so that is
nothing but the requested DT
minus which is being provided by the
cloud the remaining will
be means met out of the edge that is
DT of
e and if let us say that edge is not
providing ET is zero that is the
remaining virtual machines at the edge
at time T is zero then the demand is not
met that is the virtual machines will
not be allocated from The Edge so the
number of virtual machines which are
provided by the cloud is represented by
DTC so it should be noted that if the Ed
note has no available resources here in
this case
it will hand over all the arriving
Computing to the to the cloud for
processing so when the resource
allocation can be successfully performed
on The Edge nodes each demand processed
by The Edge node will generate the
allocation record like this so here in
the previous figure we have seen
the virtual machines which are allocated
at the edge and the duration for the
that and this demand and each Demand by
The Edge node will be generate will
generate allocation record HT so
DTE D is the number of virtual machines
which are provided by the otes in this
allocation and LT is the remaining
computation time of this particular
demand here so when a new demand arrives
the resource allocation is completed and
allocation record will be generated and
added onto the allocation record list
like this so H1 H2 and so on up to hm
which is nothing but the allocation
record so when a new demand arrives the
resource allocation is completed the
allocation record will be generated
added to the allocation record list here
this is the allocation H is maintained
the record list allocation record
list so allocation record list H is
maintain to keep track of the completion
of the user's
demand now modeling resource allocation
problem in CC so after the users demands
are fulfilled the asnes need to release
the coresponding virtual machine
according to its record and delete the
allocation record from the list Edge so
the number of virtual machines that
completed the Computing task and are
waiting to be released at the end of
time slot t
is is denoted by new T so new T is
nothing but the summation
of summation of uh d t
where where I that is
for different time
slots 1 to M such
that Li is called to zero that is hi is
an is an
H so the number of remaining virtual
Machines of the HED nodes at time slot t
+ one are the following e t + 1 is e
minus minus
D+
new so in order to quickly respond to
the users Computing demand even if no
Computing demand is found the machine is
still exhibits a standby
cost so therefore the cost of the as
mode consist of standby Cost Plus
Computing cost so standby cost of one VM
in the edge node is let us say PE and
the Computing cost of one VM in the edge
node is let us say PF so the total cost
of the of the edge node in in the time
slot T can be composed of
the that is the stand by cost that is p
multiplied by ET
plus plus the Computing cost that is e
minus
E multiplied by PF that is Computing
cost of one VM in the edge nodes and
that amounts to the total
cost that is total standby cost and
total Computing cost e minus E because
that is stand by cost is already
detected so therefore it the remaining
becomes the Computing cost so Computing
cost divided by
PF now in time slot T the cost of
collaborative Edge cloud in the public
cloud in the public Cloud environment
includes the Computing cost of the cloud
nodes and the cost of the edge nodes
which is the following
equation
so here total cost of the public cloud
is is is
given here in this case the cost of the
on demand instance for example if the
public cloud is having the on demand
instance on demand uh then let us see uh
X1 is that demand and the cost of
customization of the reserved instance
that is upfront
cost and then cost of the reserved
instance
then it is the cost and the cost of the
spot instance so here you can see on
demand then upfront customization and
reserved instance on demand instance and
the spot
instance and then the cost at the edge
nodes where XI is the service is used
one and zero is not used so the goal is
to achieve the lowest operational cost
under different strength of demand
demanding amount and the computation
time
requested now therefore resource
allocation as a decision problem we have
to see decision or the action denotes
the ratio of virtual machines that will
be provided by The Edge nodes and the
cloud nodes so a of
T that is the decision or the action
a so comprises of XC and XK for the
private Cloud scenario the there is the
virtual machines which XC means virtual
machines which are provided by the EDG
notes called XC XK means the virtual
machines which are provided by the
private Cloud now at that is the action
part for public Cloud here it Remains
the Same XC that is the virtual machines
which are provided by the Ed notes where
the virtual machine which are provided
by the public cloud is
parameterized where XK is the virtual
machine provided by the public Cloud but
K
becomes which type of service of a
public cloud is taken that is what is
parameterized that is K is
k1 belongs to K1 K2 and K3 K1 is on
demand instance
service and K2 is let us say reserved
instance service and K3 is a spot
instance
service so let us consider an
example where the the users submit the
following demand in three consecutive
time
slots see at the time slot
one like
here time so this is the first one now
this is the time slot let us say zero
and here it is one here two here
three here four here five this is
T now it's says
that uh in the time slot
one and it requires 30 so so it requires
30 and for two time
slots it requires 30
VMS and in times slot
two it requires 10 for one so in times
lot
two so it
requires for 1 * log
10 and in third time
slot third time slot now it requires
20 it requires 20 in two time
slots or two *
Lots so here
DT represent the number
of number of VMS requested and LT
represents the duration of the service
requests now assume that the time slot
one is a starting time slot so that no
VMS have been allocated diar so assume
that there are total 80 VMS present at
the edge
nodes now given constraint is that
standby cost of a VM at the edge note is
03 the Computing cost of VMS at the edge
note is2 Computing cost of the private
Cloud PC is three and unit price of on
demand instance in the public Cloud that
is p is three and it is 1.5 five that is
reserve
instance and customization cost is 800
and spot instance in the public cloud is
one now resource allocation using
private
Cloud now suppose we have our own
private cloud and a policy how been has
been deployed to allocate the virtual
machines as for
the uh client demands which outputs the
following action
so in time slot
one now action XT for K is4 7.8 here
x XK 0 and one
represents the ratio of VMS allocated
from private to the total VMS requested
the remaining VMS is 1 minus XK are
allocated The Edge now calculate the
cost of collaborative
Cloud side Computing that is private
cloud in a given instance Cloud setting
at three different time slot find the
number of VMS that will be available at
the edge node the beginning of the
fourth time slot so here if you see this
particular solution of the problem which
we have stated in the previous let us
say that ET represents the number of VMS
available at the edge at the time and
let us say that it is 80 that was stated
in the problem statement at time T1 the
demand DT is of
30 for two duration and action is that
x1k is4 so the number of virtual
machines which are allocated from the
cloud
is is nothing but X1 K multili by
D1 that
is that is point 4 is from
The
Edge 4 that is 1
minus point point that is XK is
is4 uh and multiplied by this
30 that is the demand and it becomes
it's 12 so number
of virtual machines which are allocated
from The Edge node will be D1 minus that
particular coming from from the cloud
and that is 30 - 12 that is 18 will come
from The Edge and the number of virtual
machines remaining at the edge will be
80us 18 62 so the resources can be
successfully allocated from The Edge
hence the allocation record will be
generated here
18
182 that is for the
edge so this is the allocation record so
this is the cost at the edge that is E1
* by
P plus e minus E1 * by
PF so here you can see that point if you
if you see uh in the
previous so it is the standby cost at
the edge node is let us say
03 multiplied by 62 because
62 is not used so this is stand by
cost and 80 minus
62 will be multiplied
by2 2 that is the Computing cost at the
virtual
machines
that is nothing but 5.46 is the cost of
the edge node so cost at the private
Cloud will
be the cost of the cloud Plus cost of
the
edge so that is the
12 is coming from the private cloud and
the unit cost is three and it is 5.46
that is the total cost is 41
number of
VMS to be
released in N1 is zero so the number of
VMS available at the next
slot will be the
62 now at the slot T2 the demand will be
D2 and
I2 that
is 10 and 1 for one
unit now the action
for X2 K is 7 so the number of virtual
machines allocated from the
cloud
is X K2 multiplied by D2 that is nothing
but
7 multiplied by 10 that is 7 so the
number of VMS allocated from The Edge
node will become
d 2 is nothing but the total demand is
10 D2
and the virtual machine from the cloud
is minus that that is 7 Comm three from
The
Edge now the number of virtual machines
remaining at the edge
note is equal to E2 minus D2 that is E2
is 62 that is available virtual machine
on the edge minus 3 which is required
that remaining will be 59 the resources
can be successfully allocated from The
Edge node hence the allocation record
will be generated in this manner
allocation record now will have 181 and
31 now the cost of the edge will be e2p
plus e minus E * by PF that is E2 is 59
stand by cost is
03 plus 80 -
59 that
is uh that is being given
from The Edge that
is that is 4.2 and the cost of the cloud
will be from the cloud from The Edge so
that is 7 * by 3 and 5.97 that is
26.97 and this will be the allocation
record and the number of virtual
machines waiting to be released here is
now the first time slot also will be
finished by the end of the second time
slot 18 will be released and three will
be released because and that is 21 so
number of virtual machines available at
the next next time slot that is three
will be E2 + N2 that comes out to be
total 80 will be available again at time
slot
T3 the demand will be D3 that is 20 for
two duration and the action is
X3 for X3 is k equal to8 so number of
virtual machines which are allocated
from the cloud is X3 divided by by the
demand that is8 multiplied by 20 that is
16 will be met from the cloud the
remaining is 20 minus 16 that is four
will be from The
Edge so the number of virtual machines
remaining at the edge will be 80 minus 4
that is 76 so the resources successfully
allocated from The Edge note hence the
allocation record will be generated in
this following
manner that is four for the duration two
that allocation reord and the cost of
the edge note will be X E3 P
so that comes out to be and E minus E3
that 80 -
76 that is2 that
is so uh so standby cost plus the cost
of the edge comes out to be 3.08 and the
cost of the cloud will
be now DCP
C that is 16 multiplied by 3 plus the
cost of the edge is 51 so total cost is
51.8 and the allocation number of VM
waiting to be released in N3 is equal
to0 so number of virtual machine the
next slot will be
76 now similarly we can see the resource
allocation using public Cloud setting
with a new policy that outputs the
following time slot now here it is
parameterized public Cloud now here you
have
KT and XT where KT
is parameterized whether if it is zero
then it means on demand if it is one
zero means on
demand one means it is
reserved
instance and two means a
spot
so now the query is to calculate the
cost of collaborative Cloud side
Computing that is for the public cloud
in the given public Cloud setting at
each of three time slots assume that se
demand were met by the client and that
no customization is performed on the
reserved
instances so at time T1 the cost
calculation for the edge node will
remain the same as calculated in the
private Cloud so the cost at the edge
node from the previous case that is C1 P
plus e minus E1 that is PF is
5.46 the cost of the
edge public cloud is C1 public then
D1 of a and then it is considered cost
of the reserved instance only here in
this case and plus C1 e so that is
calculated over
here now for time slot
2 that particular calculation is
again that is E2 P plus e minus E PF and
that is nothing but a public Cloud that
is the cost of the on demand instance
only since since the time slot two is uh
allocated for on demand instances comes
out to be
26.97 now coming to the third one here
the third one is the spot instance and
that cost comes out to be
19.08 now long-term optimal resource
allocation decision making we have seen
that the computation cost of the cost
function at each step however we would
like to minimize the long-term optimal
resource allocation objective or t steps
so here T is equal to one to T steps and
this is the cost of private and public
so so we shall model the resource
allocation problem as Marco
decision problem which converts the cost
that is minimization problem into an
equivalent
reward maximization problem so Marco
decision process is a discrete stock
estic Control process it provides a
mathematical framework for modeling a
decision making process where the
decisions were made at each discrete
time steps mdps are useful for studying
optimization problems soled bya dynamic
programming and reinforcement
learning so Marco decision process mdp
is a five tble s a p r r where s means
set of
states and a means the set of
actions p is the
state multiplied by by the actions
multiplied by
state so that means given a state given
an action it will make a
transition and another state is
represents the transition probability
with PS being the probability
distribution over the next state so PS
s by S comma a is the probability of
making the transition from State S
Prime from
s through the action
a so R so that
was the transition probability R is
represent the reward function so reward
function again is s by a by S and it
gives a reward function where RT is R of
St at and St + 1 being the reward
obtained by moving from State St to St +
1 y
the action and
row represents the initial State
distribution additionally S Prime
represents the set of final States so
the name Marco decision process refers
to the fact that the system obeys Marco
property that is the transition only
depends on the single most recent State
and the action pair and no prior
history now using the Marco decision
process let us understand about
reinforcement learning so reinforcement
learning the environment agent
interaction is represented
by Marco decision process where the
agent can observe the environment that
is State
s State s and take the decision that is
action
e so objective is to learn the optimal
policy that maximizes the expected
return or some of the so type of
policies are deterministic policy and
stock estic
policy so reinforcement learning that
decision process let us take this
particular trajectory at the beginning
of the decision process at time t equal
to zero the environment assumes one of
the Estates from the initial State
distribution and during each step T
observe the state s decide the action
that is at and make a transition to the
new state with the
probability to reach this St + one new
state receive the
reward and terminate if the final state
is reached so obtain the trajectory that
is S1 A1 R1 S2 A2 R2 St a RT and so on
up to St + 1 discounted trajectory
return is nothing but RT means the
reward
function at time T and and then it is
the
gamma so the probability of the
trajectory is the probability that an
agent using the policy encounters a
trajectory
T so the probability can be derived
as as this this transition on S1 with
the with the policy
pi and the probability of
S2 reaching from S1 on an action a
multiplied by again the policy and then
probability S3 and so on so it will be
total the
multiplication or a product dot product
of
this particular
policy and the
probability and expected return if you
can measure out of that particular
trajectory probability of a trajectory
it comes out to be
expectation so reinforcement learning
objective is to find an optimal
policy that is p star that maximizes the
expected return so what is that policy
what is that
policy that
maximizes so that is nothing but an AR
Max so that set of
policies which gives the
maximum return
okay now let us consider the value
function State value functions nothing
but an expected return starting at State
s so these are defined as an
expectation
of
that trajectory function drawn for that
particular policy that is Si equal to s
that is starting from then State action
value function expected return is
starting from the state s and taking
action
a given by the Q States Q values q s
comma e that is nothing but an
expectation Advantage function is how
better is to take that action a in the
state s compared to the other actions so
that is an advantage function is the Q
value of sa minus V of ss that is State
value function that means how much it
has now Q value function on this optimal
State action value function so the state
action value function that gives the
maximum expected return is starting at
State s and taking the action a and then
following the optimal policy
thereafter that is the
optimal Q values of SE so optimal policy
is the policy that will always select
whichever action is the best maximizes
the expected
return is starting at State s so the
relation between Optimum policy and
optimal State action value function we
obtain the optimal action a star using Q
Star as follows a star is nothing but AR
Max Q star sa now coming to the bman
equation for optimal State action value
function that ban function is the
optimal State action value function is
nothing but an expectation
of uh
of an reward essay and the gamma times
that maximum of Q values Q value
function that
is s prime a
prime now let us take this example of a
q value
function through this particular example
now here the mdp states are a b c d e f
it's a five
reinforce it's a five tle that is the
states are a b you can see 1 a b then C
D E F these are the set of states the
action are two actions A1 and A2 the
discount Factor gamma is equal to 1 and
the transition is deterministic so let
us see about the dynamic programming you
starting from the last states where the
Q value of final State f with star is
equal to zero now uh so go to each
predecessor node and update the Q values
using Q star s comma a with using this
bman equation so the remaining so you
can see here now uh that Q value of D
and Q value of e is calculated
as as 1
plus here can 1 + maximum
of uh the qy values of
fa so so that becomes zero that both of
them will become one here in this
case now after reaching d and e
computation now go to the predecessor
node and update the Q values using that
bman equation which is shown now here
for there are two B and C we are going
to calculate
for for the remaining B and C so for Q
value of B now is action if it is taking
the action
A1 that is
4+ this Q values and if he is taking the
action let us say A2 that is 3 + + 1 5
and 3 similarly QC so so you can see
that this is having the higher value so
it will be retained similarly for C
there are two actions A1 and
A2 now
A1 will give the six plus the maximum
gives you the value seven Q value and Q
value by action two is only three so it
will be
selected so you can see
that the action will be for if it is in
B action will be A1 if it is C the
action again will be
A1 now we going to
predecessor a so for a there are two
actions 2 and
one so if it is taken
A1 then the value is seven and if it's
taking the A2 that the total value
become eight so it will take this
particular so here you can see that the
optimal
policy is AR Max and therefore the
trajectory you can see it will take
A1 and then A2 so A2 A1 will take from a
from a it will take the state C and from
C it will take the action A1 from C it
will take the action
A1 and
it will reach the next state
D will and from D it will
take the action to reach F and for that
it will
take the action
a that is any action if it takes it will
it will reach so
Total return is 1 6 + 8 here in this
case now Q learning if you see will
update the
rule that is qsa is nothing but 1 minus
Alpha sa + Alpha r + gamma times the max
of Q values so Q learning algorithm if
you can see can be represented here that
will require the behavior policy using
random learning rate is Alpha is between
0 and one and discount Factor gamma is
between 0 and one so initialize the Q
Table stores the Q values of each state
action pair for end rounds it will
explore the Observer State s and choose
the action a using the behavior
policy e and receive the reward R and
transition to the next state will update
using the bman equation so there are two
things one is called exploration the
other is called exploitation and it is
controlled by the behavior policy that
is p
p so Q learning implementation we shall
use this implementation to solve the
optimal resource allocation
problem and it uses the ABS silent
greedy exploration the the behavior
policy chooses the action based on the
exploration probability ABS silent
during the learning phase and absil
greedy Behavior can be
implemented as shown in that particular
code now mdp formulation of resource
allocation now coming back to the
resource allocation for the sake of
Simplicity we shall assume a private
Cloud scenario only furthermore we shall
discretize the action to make it a
finite set the mdp can be stated so with
the states keeping e t DL that is the
discrete set of four
tles where e is drawn from uh 0 to e
represents the number of
available Edge virtual machines t means
represents the current time slot and D
represents the number of virtual
machines demanded by the user and L
demands represents the duration for
which the user can occupy a demanded
virtual machine all 1 2 3 four these
four parameters becomes the
state so state is nothing but the four
TPL and these four toples will have four
different parameters in the state
together Define the state now action a
is 0 point 0.1 point2 and so on up to
one so between 0 and one the actions are
there so discrete sets contains 11
actions and each representing the ratio
so these are 11 and each represent the
ratio of VMS allocated from the private
Cloud reward St at and St + 1 equal to
minus the cost function of the private
cloud is is the reward function is the
negative the cost value and initial
state of mdp start from the state S1
where S1 is equal to E1 D1 L1 where D1
and L1 are sampled uniformly from within
the the range one to D and final State
St is reached after the T time steps so
transition function is stochastically in
our mdp comes from the random user
demand we can assume the user to demand
uniformly from the set of two tles that
is D and L so D is from 1 to D and L is
from 1 to L in the next case in this
case the probability of the next user
demand has uniform distribution over the
set
D so mdp formulation of the resource
allocation assuming that the number of
VMS at the edge is 19 and the user can
demand the maximum D is equal to 4 VMS
for a duration of at most L is equal to
three time slots allocation decision has
to be made over tal to 15 time slots we
have the total of x equal to TDL e + 1
that is 15 * 4 * 3 * 19 + 1 that is
different possible mdp so total number
of possible user demand DL is 12 hence
12 possible initial States one of which
each of the user demands are set of
possible user demands is given by this
particular
D so you have to run the Q learning here
in this case so Q learning with Ile and
greedy exploration and then you
implement the policy using argmax Q
values and you have to test the learn
policies and will obtain the
trajectories and find the minimum cost
at the private Cloud over the T
steps now coming ddpg for private
Cloud resource allocation so deep
deterministic policy gradient algorithm
is the policy optimization method based
on the classical actor critic style
reinforcement learning actor is the
policy Network here this
one when say policy Network me it's a
deep neural
network which implements the policy
Network so the actor policy generates
the actions based on the policies and
interacts with the
environment the
critic this is called the value
Network evaluates the actor's
performance through the value function
that guides the actor's next action thus
improving its convergence and
performance ddpg deals with continuous
action spaces so the exploration I is
managed by adding the goian noise to the
actions produced by the actor policy
Network so this a is given
by the pi Theta s +
n now for critic Network the loss
function is the
following so this loss function is
nothing but MSC loss between the
predicted Q values and the target Q
values similarly uh the target Q value
is calculated
as the following using this particular
equation using the current reward
discount factor and estimate of the
optimal Future q values actor Network
the loss function is the following so it
uses the Q value given by the actor by
by the critic Network and also using the
policy gradients so the parameter
Omega of the
actor
of the active Target Network and the
parameter Theta of the critic Target
Network are updated using the soft
update over here now P dqn for public
Cloud resource allocation the basic idea
of PDQ and as follows for each action a
in a parameterized action
space here you can
see because of X C + XK = to 1 we can
consider K and XK in the
action uh in the action function so here
you can see that the action function
contains this
particular K and x k that is Q of sa is
equal to Q of s k and XK both are
considered here where s is the states K
is of the discrete action selected in
the time slot T and XK is the parameter
value corresponding to key type of
public Cloud instance on demand reserved
instance and that we have explained
similar to dqn that is deep neural net
deep neural network is used here in PDQ
to estimate the parameterized key values
that means that the goal of PDQ is to
find the corresponding parameter Theta
that is parameter of critic Network
when Omega that is the parameter of
criting network is fixed it can be
written as the
following so the loss function of the
value Network can be written using this
equation and in the similar manner the
loss function of the policy Network can
be written by this equation here so
training steps the input of the
algorithm contains the information about
the user request demands DT and the unit
cost of the spot instance in the public
Cloud at time T that we have already
seen in the previous examples in the
beginning of each iteration of the
algorithm the as note first needs to
obtain the state as of a collaborative
Cloud as environment and then pass the
state as the input of the neural network
into the strategy Network to obtain the
parameter value of the discrete action
all the edge node gets the action it
will it will select the appropriate
public Cloud instance based on the
discrete values in the action and
determine the number of public Cloud
instances to be used on the parameter
values then interaction with the
environment occurs to get the next state
reward termination flag storing this
round of experience to the experience
replay pool the DRL agent will sample
from The Experience replay pool and
calculate the gradient of the value
Network and the policy Network it will
then update the parameter of the
corresponding Network after one round of
iteration to ensure the convergence of
the resource allocation policy the
training will continue repeated to the
maximum number of training around sets
now coming to the Google cluster
workload traces data set which was
published in
2019 that data set contains the
workloads running on eight Google B
compute clusters for the month May 19
2019 so the traces describes the job
submission scheduling decision resource
usage data for the jobs that ran in
these clusters it has enabled a wide
range of research on advancing the
history of the art for cluster
schedulers and cloud computing and has
been used to generate hundreds of
analysis studies the new Trace allows
the researchers to EXP exp these
changes so CPU usage information
histogram for each 5 minutes slot not
just sample information about Alo uh
sets job parent information for the
master relation such as map reduce job
just as L Trace these new ones focus on
the resource request and usage and
contains new information about the end
users so the trace data is being made
available via the Google big query so
that the sophistic analysis can be
performed without requiring the local
resources this site provides the access
instructions and detailed description of
what the traces
have so if you see this particular type
of
results so uh what you can you can see
over here EF means that Ed
first and random randomly select the
edge node or a private Cloud node and
PSO is using particles form optimization
optimizes the resource allocation action
for the time so here you can see that
experimental results of a low intensity
groups so here what you will find here
is the the cost is being uh reduced here
using these kind
of uh that is Edge
first it's quite effective in all the
cases similarly when you see the public
Cloud also in the
use and then you can see that the s
first Plus on demand s plus plus random
random plus
random so uh here you can see
that the the cost uh the the lowest cost
is of the edge first
and so Edge Plus that is as first and on
demand and then other algorithms you
know that can be designed around that
particular cases so conclusion in this
lecture we discussed the formulation of
a resource allocation problem in
Cloud Edge
collaborative way of computing using
private and public clouds we have
used using Marco decision process
process problem formulation of resource
allocation problem in a collaborative
Edge cloud computing along with
parameterized dqn and the Deep
deterministic policy gradient algorithms
of a reinfor learning to solve resource
allocation problem as a decision problem
we have also briefly covered the Google
cluster workload traces data sets thank
[Music]
you
[Music]
[Music]
[Music]
