DEMO 04

IIT Kanpur July 201
20 followers
May 29, 2024
[Music]
now we shall look at an example of using
the amqp or the advaned message queuing
protocol with help of rabbit mq rabit mq
is a message broker it is available as a
Docker image we can pull this image and
run the rabbit mq message broker on our
system now I already have the
image the rabbit mq image is present on
our
system we can use this image uh to start
the message broker it will run on Port
5672 by
default so I shall use the docker run
command
provide the host name as my rabbit the
name of the container itself would be
broker and the image will be rabbit
mq it will by default connect to the
docker zero
Bridge right the container has started
let us check the docker
PS we see this Docker container with the
name broker has been started
now we can also check that in our
Network
diagram so we can see this D mq server
started at IP
address uh
17270 do2 and it is connecting to the
docker zero Bridge
so our message broker is ready we shall
first look at a very simple program that
will have a que it will have one
producer and one
consumer as you can see in this diagram
this is a simple hello world program for
rabbit mq we have a producer marked by P
there's a q that is named hello and then
there's a consumer marked by
C uh producer will produce the message
and push onto the
cues and uh the consumer will then start
consuming from the cues the Q is
essentially a large buffer of
messages and the rabbit mq the message
broker it accepts and forward these
messages now we have already started our
rabbit mq broker as a
container so we need to write a producer
program and a consumer program now we
can have have many producers that can
send message to one q and there can be
many uh consumers as well however in
this example we should use only one
producer and one
consumer so the messaging queue is based
on the amqp
protocol we can use wies shock to
capture and see the
packets so let us start the wire shock
we shall uh capture packets on the
docker zero
interface so this will capture the
packets we shall uh see this later when
we start sending the
packets so first uh I shall create the
producer and the consumer scripts now I
already have the scripts written here we
shall look at the produc proder script
first now the producer script it's quite
simple first we need to establish a
connection with our
broker now the broker as we know it is
started on the docker container it has
this IP
address the IP address would be this so
we need to specify this as the IP
address of our message broker so let us
do that
so we would have 17217 then
two now this would be our message broker
connection after establishing the
connection we can declare a queue this
will create the queue if it doesn't
exist the queue name being
hello uh and then we will simply take a
input message from the user
and publish it on this queue
but before sending we need to make sure
that the Q exists if you send a message
to non-existing Q rabit mq will drop
this
message now at this point uh we are
ready to send this
message now in rabbit mq message can
never be directly sent to a que it will
be sent to an
exchange so here we are using the
default
exchange that is identified by an empty
string this is a special exchange
because it allows us to specify exactly
to which queue the message should
go the que name is specified by the
routing
key which is hello
here now finally when we are done we can
close this
connection now coming to the receiving
side that is our consumer
consumer will receive messages from the
queue and print them on
screen just like like before we need to
establish a connection so I shall use
the same IP
address on the consumer as
well then we need to declare the
queue however it is not required to
declare it here if we start our producer
program before the consumer but
declaring it twice will not create it
twice it doesn't make a difference
here now receiving messages from the
queue works by subscribing a call back
function to this queue which is
specified by this whenever we receive a
message this call back function is
called and in this case it will just
print whatever is
received and then uh next we need to
tell rabbit mq that this particular call
back function
should be called when there is a message
from our hello
que and uh finally there would be you
know this
Loop that will run
forever and uh it will keep on consuming
until we we press this contrl C we give
a keyboard
interrupt so with that we have our
consumer and producer program ready so
let us start these
programs I will start this consumer
program
first right the consumer program has
started it is uh waiting for someone to
send to this
queue so let us start the producer as
well
right the producer program has
started uh now what we can do is first
uh
check the cues we can check what cues
have been created the the hello Q should
have been created so for that uh what we
can do is start a bass shell on our
docker
or we can start it
here so start a bash shell here now we
can
see I can use this command rabbit mq CTL
and then list
qes so we see that there is one que with
the name hello there are no
messages so now we can start sending
messages from our publisher or the
producer
so this Hello message was received here
by the consumer we can check our
packets we see that a lot of amq pack
amqp packets
there it was just starting the
connection in there and this is our amq
packet that had our message in
that e
so like that uh we can send keep on
sending
messages using a
q so I'm going to stop this producer now
and we can exit out from the consumer as
well now we can check our rabbit mq here
again we can check our rabbit mq Q's
there is the same
que now it is showing messages to be
zero that means there are no messages to
be consumed yet however if we didn't
start our consumer and only started our
producer let's say I send the same
messages
again then these messages would be
showing up here
as two messages that are not being
consumed by the consumer yet however
once the consumer starts it can then
read this messages again it will receive
the messages and the queue will be empty
again so uh with this we have seen uh
the simple Q using the amqp
protocol we were able to send an receive
messages
using the rabit mqq now we shall move on
to the next
part we shall now look at the publish
subscribe model for rabit mq
with the publish subscribe
model we can uh deliver message to
multiple
consumers in this example we shall see
how we can do that and publish this
messages that are going to be
broadcasted to all the
receivers the idea of
exchange uh in in the message Ming model
in rabbit
mq the producer never sends any message
directly to any
Q uh it only sends it to an
exchange Now The
Exchange in turn will send it to the
appropriate cues and it will have that
information where to push which
messages now we can specify an exchange
as shown in this diagram this x is that
exchange and it has two cues here and it
can uh publish to both uh forward uh
this messages to both these cues from
Publishers now the types of exchanges
there are a few types of exchanges but
in this example they have used a fan out
exchange uh which means it is
essentially broadcasting to all the cues
that are bound to
it so we shall see this example and
create a fanout
exchange Now the default exchange that
we had used in the last example was
specified by this empty
string in that case we had specified the
routing key which was the Q name but
while using exchanges we don't need to
specify routing key instead we shall
specify the exchange
name so this is the producer code and
the consumer code from the last example
or the last demo uh but this time we are
going to use this publisher
code it is it is almost the same but uh
the changes that we can note here is
instead of declaring a q we are
declaring an exchange here we are
providing the name of the exchange and
the type of the exchange in producer we
had specified Q declare and given the
name of the
queue uh the rest of the program is uh
almost the same we again in the publish
method we use the name of the exchange
instead of routing
key and similarly for the Subscribe the
subscriber we shall have the following
code again we shall establish this
connection and declare The Exchange
and the
type we shall also declare this
temporary
Q the name is empty that means that name
would be randomly assigned and this
queue would be bound to our
exchange and then to consume the
messages from this
queue uh the method is the same the
basic consume
method we will have a call back function
that will handle these messages
so with that let us uh start our
publisher and uh subscriber
programs I will first start
the subscriber program uh sorry I'll
first I will first start the publisher
program uh we also need to start the
docker
container so let us first start the
docker
container the container is
started now we can uh start our
publisher program first
the publisher has started it is waiting
for us to input a message we shall also
start the subscriber
program yes it is started as well so if
we send a message here say
hello it's receiv receiving that hello
message now we can have multiple
subscribers here so I can start another
subscriber now this subscriber is also
assigned uh sorry it's also consuming
from the same queue or the same exchange
so it shall also receive the same
message so let us say we
type World
here so now both the exchanges are sorry
both the subscribers are receiving the
message so this was a very simple
example of single publisher and multiple
subscribers using
Exchange now we shall look at uh another
example uh with the simulated sensor
data so we can create some sensor data
and use
this amqp
protocol in this uh publish subscript
fashion our sensors will would publish
the data and there would be subscribers
which which would be for example
programs like that store data or analyze
them or
control so let us see an example of this
I'm going to stop these
now I have a simulator
here now I have a robotic simulator in
which uh there is a
drone this drone has a variety of
sensors uh built into that including GPS
sensor so we shall uh take the GPS data
from the sensor of this drone and send
it over this exchange send it to The
Exchange and then there would be
subscriber programs that would be
subscribed to this data and they will
carry out different
activities using that data
now I have already added this part of
code in the controller of that uh
drone it has a python controller I've
already added the code to publish uh to
connect to the
proper uh uh rabit mq broker to
establish the connection and then uh
declare the
exchange and then uh it can be uh then
it can and publish messages so we can
use this publish
message
function and it
shall and it shall publish this uh to
the required
exchange now on the receiving side we
can have some program that will receive
uh the data from this drone so let us
see an example of this such kind of a
program first I will create a subscriber
program that will just uh take the data
from this drone and store
it I will call it
sub
drone the sub drone data the subd Drone
program is first going to establish the
connection then I'm going to then
declare The Exchange
we are also going to bind uh the Q
name a random Q
here and the exchange name will be
drwn we have specified The Exchange name
here as
well the publish host
being 1721 17.0.2
there is that is where our Docker
container for rabbit mq is running the
publish Q name or the exchange name has
been described as drone and we are using
the same exchange here as
well now we need to provide a call back
function so this would be our call back
function what it will do is simply uh
open a new new file when the Drone joins
or the Drone is
started and it will write messages
whatever it receives from the Drone that
drone would be sending its GPS
coordinates that would be X Y and
Z and that data would be written inside
this drone log. text
file by this call back
method now we can
uh specify the consume
the consume method so we can specify
this Q name and this call back
method after that we can call the start
consuming
function now our uh subscriber is ready
it's going to read that uh data from the
GPS data from the Drone and write it to
the log file so we can run this one and
uh see if it is working
sub drone.
py right it has started it is waiting
for the data to come from the sensors
that is our drone so let us also start
our
simulation so we shall see this
simulation and our output side by
side we can exit from this one
so it has started it has opened a new
file for writing the log now these are
the coordinates the GPS coordinates
being sent by our uh drone
here so we can move it around a bit and
see how this program is going to record
all these
coordinates so that's enough I'll stop
it now
okay it has stopped the controller we
can pause the
simulation however it is still running
uh if we start this simulation again or
this uh subscriber starts receiving from
this uh data from this drone again it
will keep on recording that however I'm
going to stop it
now now we can see this drone log. text
that these coordinates
were recorded by the our program that
was our
subscriber we can also try to visualize
this so I'm going to plot this
data we shall
see so this was the path the trajectory
followed by our
Dr that was recorded by our uh
subscriber
so
now since this was a fan out exchange we
could have multiple subscribers
here so let us create another
subscriber which is uh also going
to control this drone
so for that uh I can create another uh
subscriber
here I'm going to call it C and drone
for
control so the same things
here but we are going to Define an
additional que that would be for sending
the actions to our drone
so that queue would be action we are not
using an exchange here we are using a
simple que because only this uh program
is now acting as a publisher and the
Drone is acting as a
subscriber so with this uh the Drone can
read from the action queue and take the
action as uh said as sent by this
program so we shall uh
create this uh connection as well now
the second connection is uh for
subscribing the data from the Drone
which is through the exchange that is
our drone exchange so we can mention
that here as
well now this connection is for reading
the data the GPS data from the Drone it
will read the data and based on the data
it will take an action and send that
action to the action Q which the Drone
is subscribed
to now I shall create this call back
function right so this is the call back
function so what it is going to do is uh
check uh check the Z coordinate of the
data of the Drone so that would refer to
the altitude of the
Drone so what it does is simply checks
if this altitude is too low that is it
is less than uh 2 m or if it is too high
that is greater than 2.0 M then it is
going to issue an altitude
warning and it will also send a
message to the action Cube that is to
take the corrective action that is
either to increase the altitude if it is
going too low or to decrease that
altitude if the Drone is going too high
so it is going to publish on the actions
Cube which the Drone can
read so with that uh this program is
ready
now we need uh another program that is
uh the action has to be taken on the
Drone because we didn't uh write any
code to receive actions from this action
CU so for that I'll create another
file I will call it act
drone now here I'm going to use uh this
special function called control drone
altitude it is already
written uh in the library that is
available with the Drone so we can use
this function to control the attitude
directly now here we need to establish a
connection to the actions
CU then we Define our call back and now
the call back would be completely
handled by the control control altitude
drone altitude and we shall also also
print the
message so with that we can start
consuming the messages from
here right so our uh act drone is ready
that is going to control the altitude
the con drone the control drone is
simply going
to uh give us an warning and publish
this action which the Drone can
read and this act drone is actually
going to run uh it's going to
communicate with the process of the
Drone controller so it can directly
control its altitude from here so we can
start
these all these uh
subscribers so I'm going to reset the
simulation and then I'm going to start
to the sub drone first which is going to
again log the
log the GPS positions let us delete the
existing
data so I shall start this uh subd drone
first right the sub drone is
started now I will also
start the other one that is control run
right this process is started as well
and finally the ACT
drone right all the three scripts are
now started these are waiting for the
simulation to run and the data to come
from our
drone so let us
see so now I'm going to start this
simulation so let us start the
simulation right new loog file has been
created now this is the control run now
control rone is not taking action
because
the uh the altitude is normal now and
that is what is outputed here so if we
try to change our
altitude let's say we move ahead a
little bit and then try to go too
low as you can see it is it is giving a
warning and then try to increase it back
to normal uh the same thing will happen
if I try to increase my
altitude too high so say it's saying
decreasing altitude so it is
automatically decreasing the altitude
I shall stop this demonstration
now we can also stop our subscribers
we can see this drone log was created we
can try to visualize this
again so this was the data that was
generated by our drone
flight the path of the flight it was
recorded by our
subscriber similarly our control drone.
py this subscriber was also able to
control the Drone and stabilize its
altitude so this way we have seen that
we can use this rabbit mq and this
messaging and queuing systems to uh
collect data from our sensors and
control them this has to
be this is all possible because of low
latency network
connections and uh where lightweight
protocols the mqp
protocols so with that let us move on to
the next
part now we shall look at some machine
learning models specifically the Deep
neural networks for intelligent
Edge now object detection is an
important application of computer
vision there are many models for object
detections such as rcnn and
YOLO we shall look at one such
model that is called YOLO uh with an
S YOLO uh was proposed in this
paper it stands for you only look at one
sequence so it's an implemented of
object detection uh through vision
Transformers viit has been uh already
proposed by
authors so they have used the vision
Transformers in place of convolution
neural
networks now this uh Transformer model
uh it divides that uh input image into
patches which is uh treated as a
sequence and then fed into the
encoder which then uses uh this a
Transformer
architecture and it uses it to predict
the bounding boxes the B boxes
here and the
classes so we we shall look at this
model there are various different models
such as ulos for image processors
feature
extractors but we shall look at uh the
one for object
detection now this model is available on
the hugging face website for
Transformers we can get a pre-train
model here uh this is based on Pyon
TCH so there is an example given here on
how to use this Yolo's
model so they have used uh the Yolo's
tiny
model we shall also use this tiny model
and uh use it to use it for inferencing
so let us see with this example how we
can do
that so I will create a new notebook
file
here and then we can use this code
existing
code so make the Imports
first now we need to define the the
image processor and the model we shall
use this Yolo's tiny model
right the model will get downloaded from
the
internet however I have this model
already so it's going to use it from the
cache and then uh we also need to Define
our
image that is the input
image so the input image is going to be
taken from this URL we can see what this
image looks
like so this is a image of two cats
lying on a couch there are two remotes
as
well so now we
can input this
image into our model
and it will produce the outputs now we
need to process the outputs as
well so we shall use this piece of
code
right so it's also going to uh tell us
the bounding boxes the location and the
uh the size of of the bounding
box so we we shall see the output of
this uh this
model so it has detected a total of five
objects out of which three are
remote although there are two remotes
here and there are two cuts and these
are the bounding boxes now we can also
draw this bounding boxes
this will plot the bounding boxes and
also annotate the object with the class
label so let us uh see this bounding
box uh I forgot to import this
one I'm still getting some
error right uh we have not
defined item here let me redefine this
one
yes yes so now the bounding box are
showing up so it has detected two
remotes on the same one there's a remote
detected here and the cat detected here
the other cat detected here so we can
look at a few more
examples a few more example
images so I have some images here
this is a image of a person sitting at a
office so let us see what our model
detects so our model has detected four
objects here two of which are ported
plants three are ported plants and one
is a person as we can see in this image
we can look at one more example
here let us try some other
image this is a image of two calls let's
see what our model outputs
here it has detected two calls
correctly now we shall look at one last
example model has detected a
bus so many cars there's a clock
detected here and a person detected
here so this model Works quite well
fairly
well uh so we can use this
uh uh model to detect objects we shall
look at an example where we shall use
this uh with our drone
earlier from the
simulation the Drone Al also has a
camera so the camera can be used the
data from the camera images so I have
this setup we have a few objects there's
a
box there's a cat there's a person a car
and a
chair now
the the controller here has already been
programmed to send
images but uh we need uh something to
handle these images and pass them
through our Yolo's model for object
detection now Yolo's model can only
detect objects in an image so we have to
extract frames from the
video and uh then pass it to the YOLO
model for detection
so let us uh create new file
here now here uh I'm going to use the
the
same model that I used here so uh the
Imports here the model you Lo
tiny and uh we shall
also listen to one of the or subscribe
to one of the exchanges the same drone
exchange that would be turning at this
IP
address uh the not not the Drone
exchange but the rabbit mq broker would
be running at this IP address and there
would be an exchange called drone and uh
previously the Drone exchange the Drone
was publishing its GPS data on the
exchange but this time it's publishing
its images the camera
images and we have to read those camera
images from our script or our
program and pass them through the YOLO
model to
detect the images the
objects so for that we already have
defined this exchange we need to define
a call back that is going to process our
data so let us uh Define this call
back so this is our call back uh I've
used the same code from the previous
one for detecting the object and drawing
bounding boxes around
them and this is what H happens in the
call back the detection will happen in
the call back and it will
output the image with the bounding boxes
and this is going to happen as long as
there are messages to be consumed that
is the images are being sent by the
Drone so our uh consumer here is ready
with the YOLO model inside
it and it can start consuming we can run
this one
right our consumer has started it is
waiting for the Drone to transmit
transmit its uh
data so let us also start our
simulation right now we need to switch
on the drone camera the camera is on
uh and we see here that we have started
receiving these images from the
Drone now it is not able to detect this
box
properly however we shall move this
drone around and see if it is able to
detect the other objects or
not so I'll slow down down the animation
or the simulation and
turn this drone around uh
slowly right our drone has started
turning around let us see if it is
detecting the other objects or
not so I shall scroll to the bottom here
right here we see that this cat has come
into the frame however it has been
detected as a
cow oh this dummy is being correctly
detected as a
person and then the car is coming to the
frame it is also being detected
correctly as
car so as our drone turns around it is
detecting all the
objects this chair has been detected as
a
bench it's not correct however it is
closed so now uh it has moved to the
other view we can turn this drone around
again we can uh wait for it to turn
around again and uh go for a second
route of
detection we can try to increase its
height a little
bit to get a different
view we should turn back a little
bit and look at it from an higher P
perspective
let's see if there's any changes in the
deduction uh the cat is out of frame it
is not being detected correctly however
this person is being detected
correctly now the cat again being
detected as the
cow the car is been detected correctly
however its wheel has been detected as a
clock now we did not uh train this or
fine tune this YOLO model on our data
that is the images from our
camera now although this is simulation
these objects might not accurately
represent uh the training
data however our model is uh still
performing fairly well it is able to
detect some objects such as the car and
the person it has detected correctly now
now I'm going to stop this uh
simulation right the simulation has
stopped now we can stop our subscriber
as
well so in this way we have seen uh the
working of Yolo's
model we inst install this ulo model on
our drone not exactly drone the ulo
model was in fact running uh on the edge
that is our
host and it was communicating
through through through that Docker
Bridge Docker zero Bridge with our
broker that is a rabbit mq and this
drone was sending all its data
the camera feed through the rabbit
mq cues and that was being received here
by our subscriber that was running on
the edge not on the Drone itself and it
was running this YOLO and it detected uh
whatever it
could and with that we have seen this y
model now we shall look at some more uh
such
models in the next
part so with that uh let us end this one
here I'm going to exit out of this
simulation
[Music]
[Music]
[Music]
