added test diagrams, gantt chart, sent to Ning

This commit is contained in:
aj 2020-05-13 17:00:36 +01:00
parent 3a1dacd6da
commit 19d40c5a4c
7 changed files with 283 additions and 135 deletions

BIN
GanttChart.ods Normal file

Binary file not shown.

View File

@ -245,10 +245,10 @@ LiveScan3D
\noun default \noun default
suite is presented, facilitating the acquisition and presentation of multiple suite is presented, facilitating the acquisition and presentation of multiple
scenes simultaneously. scenes simultaneously.
In doing so the application suite already strengthened by it's domain-agnostic In doing so the application suite already strengthened by its domain-agnostic
nature can be seen to be further generalised allowing conference call-like nature can be seen to be further generalised allowing conference call-like
experiences. experiences.
Developments were made to the server display component allowing this concurrent Developments were made to the server display component to allow this concurrent
presentation and to the network layer of the suite in order to differentiate presentation and to the network layer of the suite in order to differentiate
sources. sources.
Extensions to the Extensions to the
@ -460,11 +460,11 @@ This project aims to extend this suite to support multi-source holoportation,
\begin_layout Standard \begin_layout Standard
One application would be support for experiences akin to conference-calls One application would be support for experiences akin to conference-calls
with multiple actors capturing their own environments for composition at with multiple actors capturing their separate environments for composition
the server. at the server.
This could be seen to be applicable both to productivity software similar This could be seen to apply both to productivity software similar to existing
to existing conference-call software and to entertainment experiences with conference-call software and entertainment experiences with multiple locations
multiple locations combined for consumption by the public. combined for consumption by the public.
\end_layout \end_layout
@ -594,7 +594,7 @@ status open
\begin_inset Graphics \begin_inset Graphics
filename ../media/premise.png filename ../media/premise.png
lyxscale 30 lyxscale 30
width 70col% width 60col%
\end_inset \end_inset
@ -637,17 +637,6 @@ Objectives
\begin_layout Standard \begin_layout Standard
In order to achieve the goal of multi-source holoportation the following In order to achieve the goal of multi-source holoportation the following
key objectives must be achieved, key objectives must be achieved,
\begin_inset Flex TODO Note (inline)
status open
\begin_layout Plain Layout
Need more explanation of the objectives below, in particular what is the
technical challenge
\end_layout
\end_inset
\end_layout \end_layout
\begin_layout Enumerate \begin_layout Enumerate
@ -658,11 +647,15 @@ Introduce a source ID in network communications to identify each frame of
\begin_layout Enumerate \begin_layout Enumerate
Migrate the server from a single-source streaming node to a manager of multiple Migrate the server from a single-source streaming node to a manager of multiple
scenes including a method to synchronise sources. scenes including a method to synchronise sources.
This includes defining methods of identifying sources that are no longer
actively transmitting frames to remove them from presentation.
\end_layout \end_layout
\begin_layout Enumerate \begin_layout Enumerate
Extend the native viewfinder of the server in order to separately render Extend the native display viewfinder of the server in order to separately
each connected source. render each connected source.
Allow individual placement of holograms using keyboard controls.
Include automatically coherent placement of newly connected sources.
\end_layout \end_layout
\begin_layout Enumerate \begin_layout Enumerate
@ -672,6 +665,7 @@ Android
\noun default \noun default
AR application to separately present each source of footage and allow individua AR application to separately present each source of footage and allow individua
l control. l control.
This would represent multi-source developments to an AR presentation layer.
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -686,6 +680,13 @@ noprefix "false"
\end_inset \end_inset
. .
The greatest challenge to a seamless multi-source experience is that of
the previously mentioned synchronisation.
While methods to achieve temporally matched presentation are investigated,
these have the potential to significantly increase complexity in the processing
pipeline.
This makes the design of these systems critical in maintaining efficient
real-time processing.
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -766,8 +767,8 @@ e capabilities and some elements of the mobile AR display.
\end_layout \end_layout
\begin_layout Enumerate \begin_layout Enumerate
Alter the AR environment of the mobile application in order to allow deployment Migrate the AR environment of the mobile application in order to allow deploymen
to both t to both
\noun on \noun on
Android Android
\noun default \noun default
@ -779,6 +780,13 @@ Implement a dynamic system of frame rate throttling to investigate the effect
on effective display latency. on effective display latency.
\end_layout \end_layout
\begin_layout Standard
The biggest challenge to the mobile AR migration is the scope of changes
being made to the codebase, changing the library used to create the AR
environment.
This is a significant change and poses many opportunities to cause issues.
\end_layout
\begin_layout Subsection \begin_layout Subsection
Key Technical Achievements Key Technical Achievements
\end_layout \end_layout
@ -788,20 +796,6 @@ This section summarises the design and implementation of the developments
made within this project. made within this project.
\end_layout \end_layout
\begin_layout Standard
\begin_inset Flex TODO Note (Margin)
status open
\begin_layout Plain Layout
minimum two pages to highlight/summarize what you have done, including your
design, implementation, testing strategy and key observations.
\end_layout
\end_inset
\end_layout
\begin_layout Subsubsection \begin_layout Subsubsection
Multi-source LiveScan Multi-source LiveScan
\end_layout \end_layout
@ -888,12 +882,12 @@ Kinect
\begin_layout Standard \begin_layout Standard
The mobile AR application was updated in order to allow the simultaneous The mobile AR application was updated in order to allow the simultaneous
display of multiple sources. display of multiple sources.
Utilising the strengths of Utilising strengths of
\noun on \noun on
Unity Unity
\noun default \noun default
development, this did not require a significant redesign of the written development, this did not require a significant redesign of the written
code, instead the hierarchy of game objects and components could be altered. code, instead, the hierarchy of game objects and components could be altered.
The The
\noun on \noun on
PointCloudRenderer PointCloudRenderer
@ -1167,11 +1161,7 @@ ts outlined.
\begin_inset Newline newline \begin_inset Newline newline
\end_inset \end_inset
In support of ongoing research into the Section
\noun on
LiveScan
\noun default
suite and its network behaviour, section
\begin_inset CommandInset ref \begin_inset CommandInset ref
LatexCommand vref LatexCommand vref
reference "sec:Frame-Rate-Throttling" reference "sec:Frame-Rate-Throttling"
@ -1246,6 +1236,16 @@ noprefix "false"
In the appendices, In the appendices,
\begin_inset CommandInset ref \begin_inset CommandInset ref
LatexCommand ref LatexCommand ref
reference "sec:Gantt-Chart"
plural "false"
caps "false"
noprefix "false"
\end_inset
presents a Gantt chart for the project while
\begin_inset CommandInset ref
LatexCommand ref
reference "sec:Server-UI-Additions" reference "sec:Server-UI-Additions"
plural "false" plural "false"
caps "false" caps "false"
@ -1500,20 +1500,10 @@ literal "false"
\begin_layout Standard \begin_layout Standard
A visual hull defines a 3D representation of an object constructed through A visual hull defines a 3D representation of an object constructed through
the volume intersection of multiple 2D silhouettes. the volume intersection of multiple 2D silhouettes.
With knowledge of the relative positions of each view point, corresponding With knowledge of the relative positions of each viewpoint, corresponding
silhouettes of an object can be triangulated to form a 3D object. silhouettes of an object can be triangulated to form a 3D object.
In doing so depth information lost in projection to a 2D image can be reconstru In doing so depth information lost in projection to a 2D image can be reconstru
cted. cted.
\begin_inset Flex TODO Note (Margin)
status open
\begin_layout Plain Layout
diagram?
\end_layout
\end_inset
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -1593,7 +1583,7 @@ literal "false"
\begin_layout Standard \begin_layout Standard
The strength of these techniques draws from data gathered from tightly calibrate The strength of these techniques draws from data gathered from tightly calibrate
d camera viewpoints, however when considering consumer-grade applications d camera viewpoints, however, when considering consumer-grade applications
this is typically not available. this is typically not available.
The use of infra-red structured light and The use of infra-red structured light and
\emph on \emph on
@ -1705,7 +1695,7 @@ literal "false"
\begin_layout Standard \begin_layout Standard
Here 3D conference calling of the type described in the introduction without Here 3D conference calling of the type described in the introduction without
AR or VR applications is presented, instead users watch a composite conference AR or VR applications is presented, instead, users watch a composite conference
space on a screen with all participants rendered within. space on a screen with all participants rendered within.
\begin_inset Note Comment \begin_inset Note Comment
@ -2234,7 +2224,7 @@ Android
\begin_layout Standard \begin_layout Standard
The advancement of mobile AR bolstered by the growing ubiquity of compatible The advancement of mobile AR bolstered by the growing ubiquity of compatible
hardware has led it to becoming a rapidly growing and popular form of XR. hardware has led it to become a rapidly growing and popular form of XR.
The introduction of OS-level SDK's in The introduction of OS-level SDK's in
\noun on \noun on
Google Google
@ -2900,7 +2890,7 @@ ed.
LiveScan3D LiveScan3D
\noun default \noun default
has two associated AR clients of which the most relevant to this project has two associated AR clients of which the most relevant to this project
is the mobile AR client, subsequently the current SDKs facilitating AR is the mobile AR client, subsequently, the current SDKs facilitating AR
development were presented and their relation to the development were presented and their relation to the
\noun on \noun on
Unity Unity
@ -3603,7 +3593,7 @@ literal "false"
\noun on \noun on
Unity Unity
\noun default \noun default
and included buffers inline with the developments discussed in section and included buffers in line with the developments discussed in section
\begin_inset CommandInset ref \begin_inset CommandInset ref
LatexCommand ref LatexCommand ref
@ -3959,8 +3949,8 @@ In order to maintain a strength of the suite in it's agnostic behaviour
to different display methods, it was made critical to the design of the to different display methods, it was made critical to the design of the
OpenGL extension that spatial processing would be specific to this display OpenGL extension that spatial processing would be specific to this display
only. only.
Mobile AR applications will arrange and interact with holograms in a different Mobile AR applications will arrange and interact with holograms differently,
way, as will head-mounted AR or VR applications. as will head-mounted AR or VR applications.
In maintaining this separation of network and presentation layer each display In maintaining this separation of network and presentation layer each display
can operate independently and no assumptions are made as to how a display can operate independently and no assumptions are made as to how a display
method should handle each hologram. method should handle each hologram.
@ -4447,7 +4437,7 @@ Each source is assigned a default transformation which can be overridden
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
Sources are initially arranged in a circle around the origin in the center Sources are initially arranged in a circle around the origin in the centre
of the space. of the space.
This is done by retrieving a transformation from the This is done by retrieving a transformation from the
\noun on \noun on
@ -4708,8 +4698,8 @@ Within the
\noun on \noun on
LiveScan LiveScan
\noun default \noun default
suite two network communications required updating, those between the client suite, two network communications required updating, those between the
and server and those between the server and user experiences. client and server and those between the server and user experiences.
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -4801,7 +4791,7 @@ Implementation
\begin_layout Standard \begin_layout Standard
An integer source ID was created to be included in communications throughout An integer source ID was created to be included in communications throughout
the suite. the suite.
For conciseness this was represented by a single byte during transmission For conciseness, this was represented by a single byte during transmission
resulting in a range of possible source IDs between 0 and 127. resulting in a range of possible source IDs between 0 and 127.
Considering the use case of a source representing a scene and the bandwidth Considering the use case of a source representing a scene and the bandwidth
required for each, 128 possible concurrent sources was deemed sufficient. required for each, 128 possible concurrent sources was deemed sufficient.
@ -4958,9 +4948,9 @@ ce.
At a high level, sources can be considered synchronised if frames captured At a high level, sources can be considered synchronised if frames captured
at the same time are displayed concurrently whether from the perspective at the same time are displayed concurrently whether from the perspective
of the server or a connected user experience. of the server or a connected user experience.
There are three main challenges to achieving this aim. There are two main challenges to achieving this aim.
The following assumes that timestamps captured at both the client and server The following assumes that timestamps captured at both the client and server
are synchronised using the included NTP functionality are synchronised and therefore comparable using the included NTP functionality
\begin_inset CommandInset citation \begin_inset CommandInset citation
LatexCommand cite LatexCommand cite
key "livescan3d-buffers" key "livescan3d-buffers"
@ -4990,16 +4980,6 @@ Although capturing frames at the same rate (30Hz), it is highly unlikely
Secondly, during transmission a variable amount of latency will be introduced Secondly, during transmission a variable amount of latency will be introduced
as a result of the network conditions and distance from client to server as a result of the network conditions and distance from client to server
and from server to UE. and from server to UE.
\begin_inset Flex TODO Note (Margin)
status open
\begin_layout Plain Layout
remember third?
\end_layout
\end_inset
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -5062,7 +5042,7 @@ Active
\emph default \emph default
synchronisation includes efforts to tightly match frames during processing synchronisation includes efforts to tightly match frames during processing
at the server in order to minimise the difference in latency since capture. at the server in order to minimise the difference in latency since capture.
In practice this could be achieved at the point where frames are moved In practice, this could be achieved at the point where frames are moved
from the various receiving buffers to the live buffer from the various receiving buffers to the live buffer
\begin_inset Flex TODO Note (Margin) \begin_inset Flex TODO Note (Margin)
status open status open
@ -5309,7 +5289,7 @@ s making up a hologram.
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
On each frame update the On each frame update, the
\noun on \noun on
PointCloudReceiver PointCloudReceiver
\noun default \noun default
@ -5458,7 +5438,7 @@ While this thread could be included in many places throughout the server
\begin_layout Standard \begin_layout Standard
As such the object-oriented programming concept of encapsulation was employed As such the object-oriented programming concept of encapsulation was employed
to create a object responsible for containing live frames and checking to create an object responsible for containing live frames and checking
for for
\emph on \emph on
stale stale
@ -5661,7 +5641,7 @@ noprefix "false"
\begin_layout Standard \begin_layout Standard
The use of a source ID transmitted alongside each frame provides flexibility The use of a source ID transmitted alongside each frame provides flexibility
in that processing can be decoupled from the network infrastructure itself. in that processing can be decoupled from the network infrastructure itself.
Theoretically it would also allow clients to function as multiple sources Theoretically, it would also allow clients to function as multiple sources
simultaneously, allowing the attachment of different source IDs to different simultaneously, allowing the attachment of different source IDs to different
frames frames
\begin_inset Foot \begin_inset Foot
@ -5699,7 +5679,7 @@ An alternative would be to associate a socket with a source ID, negotiated
The server display's control scheme could be more intuitive as the directions The server display's control scheme could be more intuitive as the directions
of movement are in relation to the fixed axes of the display space instead of movement are in relation to the fixed axes of the display space instead
of the view of the camera. of the view of the camera.
In practice this means that when translating objects the pose of the camera In practice, this means that when translating objects the pose of the camera
with respect to the space must first be considered in order to identify with respect to the space must first be considered in order to identify
which direction the object will be moved. which direction the object will be moved.
\end_layout \end_layout
@ -5867,7 +5847,7 @@ Unity ARFoundation
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
Following these issues a different avenue was pursued in Following these issues, a different avenue was pursued in
\noun on \noun on
Unity Unity
\noun default \noun default
@ -5933,11 +5913,11 @@ Both libraries use a session component to manage the life cycle of the AR
Unity Unity
\noun default \noun default
camera acting as the virtual representation of the mobile's rear-view camera. camera acting as the virtual representation of the mobile's rear-view camera.
In both cases this virtual camera has a pose driver attached responsible In both cases, this virtual camera has a pose driver attached responsible
for tracing the movements of the physical device in the virtual space and for tracing the movements of the physical device in the virtual space and
a component responsible for displaying the view of the rear-facing camera a component responsible for displaying the view of the rear-facing camera
as the background of the scene. as the background of the scene.
This presents the core of the environment, additionally both libraries This presents the core of the environment, additionally, both libraries
include separate components responsible for displaying discovered planes include separate components responsible for displaying discovered planes
within the space, optionally with UI directing the user to aid in this within the space, optionally with UI directing the user to aid in this
discovery. discovery.
@ -6233,19 +6213,6 @@ name "sec:Frame-Rate-Throttling"
\end_inset \end_inset
\end_layout
\begin_layout Standard
\begin_inset Flex TODO Note (Margin)
status open
\begin_layout Plain Layout
diagrams
\end_layout
\end_inset
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -6295,7 +6262,7 @@ Design Considerations
\begin_layout Standard \begin_layout Standard
When considering the final applications of these concepts in the context When considering the final applications of these concepts in the context
of a multi-source project, the ability to have a step per source was included of a multi-source project, the ability to have a step per-source was included
in order to take into account the different network conditions between in order to take into account the different network conditions between
the server and each source's client(s). the server and each source's client(s).
A single set of global FPS and delay requirements would still be provided A single set of global FPS and delay requirements would still be provided
@ -6339,7 +6306,7 @@ The existing calculations of live statistics provided the global FPS and
KinectServer KinectServer
\noun default \noun default
. .
In order to collect these values on a source-by-source basis a dictionary In order to collect these values on a source-by-source basis, a dictionary
of of
\noun on \noun on
SourceStat SourceStat
@ -6365,14 +6332,14 @@ noprefix "false"
, this iterative version was well suited to processing each collected latency , this iterative version was well suited to processing each collected latency
value. value.
Initially the values were found by collecting the latency of each frame Initially, the values were found by collecting the latency of each frame
currently within the server's buffers. currently within the server's buffers.
Theoretically this would have allowed the calculated average to be controlled Theoretically, this would have allowed the calculated average to be controlled
by the speed with which frames are moved through the buffer. by the speed with which frames are moved through the buffer.
However this was found to be ineffective as the buffers typically stay However, this was found to be ineffective as the buffers typically stay
empty. empty.
Instead a queue of fixed size was defined in order to retain the most recent Instead, a queue of fixed size was defined in order to retain the most
values of latency received at the server. recent values of latency received at the server.
This queue included calculations for both exponential and simple moving This queue included calculations for both exponential and simple moving
averages of the contents. averages of the contents.
\end_layout \end_layout
@ -6480,7 +6447,7 @@ In order to function as a proof-of-concept for future investigations a simple
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
In practice this meant that for each second a source's latency was higher In practice, this meant that for each second a source's latency was higher
than the requirement, for example, the source's frame drop rate would be than the requirement, for example, the source's frame drop rate would be
increased by 5%. increased by 5%.
The implementation in code can be seen in appendix The implementation in code can be seen in appendix
@ -6499,7 +6466,7 @@ noprefix "false"
\begin_layout Standard \begin_layout Standard
Future investigations could use a non-linear calculation in order to reflect Future investigations could use a non-linear calculation in order to reflect
the distance from which operating values are from the requirements. the distance from which operating values are from the requirements.
Theoretically this would allow the step to move rapidly in order to quickly Theoretically, this would allow the step to move rapidly in order to quickly
approach the requirements before slowing and settling. approach the requirements before slowing and settling.
\end_layout \end_layout
@ -6530,9 +6497,68 @@ noprefix "false"
Premise Validation Premise Validation
\end_layout \end_layout
\begin_layout Standard
\begin_inset Float figure
wide false
sideways false
status open
\begin_layout Plain Layout
\noindent
\align center
\begin_inset Graphics
filename ../media/TestDev.png
lyxscale 30
width 45col%
\end_inset
\end_layout
\begin_layout Plain Layout
\begin_inset Caption Standard
\begin_layout Plain Layout
Client-server relationship in a development environment located on a single
machine
\begin_inset CommandInset label
LatexCommand label
name "fig:TestDev"
\end_inset
\end_layout
\end_inset
\end_layout
\begin_layout Plain Layout
\end_layout
\end_inset
\end_layout
\begin_layout Standard \begin_layout Standard
Development was conducted with the client and server located on the same Development was conducted with the client and server located on the same
machine reducing the latency to just that of processing the frames themselves. machine reducing the latency to just that of processing the frames themselves,
see figure
\begin_inset CommandInset ref
LatexCommand ref
reference "fig:TestDev"
plural "false"
caps "false"
noprefix "false"
\end_inset
.
In order to validate the premise of the hypothesis, investigations were In order to validate the premise of the hypothesis, investigations were
made on two machines across a domestic local area network. made on two machines across a domestic local area network.
\end_layout \end_layout
@ -6619,13 +6645,57 @@ The small scale of the controlled LAN environment does not inherently incur
As the full-scene configuration requires around 275Mbps, artificial latency As the full-scene configuration requires around 275Mbps, artificial latency
was introduced as frames took longer to deliver and were produced faster was introduced as frames took longer to deliver and were produced faster
than they could be transmitted to the server. than they could be transmitted to the server.
A diagram of this testing layout can be seen in figure .
\end_layout
\begin_layout Standard
\begin_inset Float figure
wide false
sideways false
status open
\begin_layout Plain Layout
\noindent
\align center
\begin_inset Graphics
filename ../media/TestLAN.png
lyxscale 30
width 50col%
\end_inset
\end_layout
\begin_layout Plain Layout
\begin_inset Caption Standard
\begin_layout Plain Layout
Client-server relationship during initial testing in a LAN environment
\begin_inset CommandInset label
LatexCommand label
name "fig:TestLAN"
\end_inset
\end_layout
\end_inset
\end_layout
\end_inset
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
The step was specified and varied manually initially in order to visualise The step was specified and varied manually initially in order to visualise
the response in delay and FPS. the response in delay and FPS.
Following implementation of a dynamic step, the response was again visualised Following the implementation of a dynamic step, the response was again
using figure-only capture for realistic latency values. visualised using figure-only capture for realistic latency values.
\end_layout \end_layout
\begin_layout Subsubsection \begin_layout Subsubsection
@ -6644,6 +6714,17 @@ Microsoft
Azure Azure
\noun default \noun default
cloud computing environment. cloud computing environment.
This final testing scenario can be seen in figure
\begin_inset CommandInset ref
LatexCommand ref
reference "fig:TestCloud"
plural "false"
caps "false"
noprefix "false"
\end_inset
.
A A
\emph on \emph on
Standard F2s V2 Standard F2s V2
@ -6670,6 +6751,49 @@ When operating over the open internet the domestic internet connection became
As such figure-only capture was used in order to limit the required bandwidth. As such figure-only capture was used in order to limit the required bandwidth.
\end_layout \end_layout
\begin_layout Standard
\begin_inset Float figure
wide false
sideways false
status open
\begin_layout Plain Layout
\noindent
\align center
\begin_inset Graphics
filename ../media/TestCloud.png
lyxscale 30
width 50col%
\end_inset
\end_layout
\begin_layout Plain Layout
\begin_inset Caption Standard
\begin_layout Plain Layout
Client-server relationship during further testing in a cloud-based environment
\begin_inset CommandInset label
LatexCommand label
name "fig:TestCloud"
\end_inset
\end_layout
\end_inset
\end_layout
\end_inset
\end_layout
\begin_layout Standard \begin_layout Standard
\begin_inset Newpage newpage \begin_inset Newpage newpage
\end_inset \end_inset
@ -6768,7 +6892,7 @@ name "fig:Latency-and-FPS1"
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
Generally the lowering of the drop rate induces the latency to begin increasing Generally, the lowering of the drop rate induces the latency to begin increasing
at a fairly linear rate. at a fairly linear rate.
This can be seen in figure This can be seen in figure
\begin_inset CommandInset ref \begin_inset CommandInset ref
@ -6806,7 +6930,7 @@ noprefix "false"
\begin_layout Standard \begin_layout Standard
Looking to the FPS chart, the frame rate drops in response to an increase Looking to the FPS chart, the frame rate drops in response to an increase
in frame drop rate and vice versa. in frame drop rate and vice versa.
Similarly to latency a lag can be seen when changes in drop rate are received. Similarly to latency, a lag can be seen when changes in drop rate are received.
Worth noticing is that as the step drops, the FPS increases before the Worth noticing is that as the step drops, the FPS increases before the
latency begins increasing ( latency begins increasing (
\begin_inset Formula $t=70$ \begin_inset Formula $t=70$
@ -6844,8 +6968,8 @@ noprefix "false"
\end_inset \end_inset
. .
It can be seen that as the average latency increases above the prescribed It can be seen that as the average latency (blue) increased above the prescribe
latency requirement an increase in dynamic step was induced. d latency requirement (green) an increase in dynamic step (red) was induced.
As the latency fell past the requirement, the step also fell. As the latency fell past the requirement, the step also fell.
\end_layout \end_layout
@ -7106,7 +7230,7 @@ noprefix "false"
is presented to highlight the instability encountered in the cloud-based is presented to highlight the instability encountered in the cloud-based
environment. environment.
Similarly to the local environment an initial spike in latency is seen Similarly to the local environment, an initial spike in latency is seen
before reducing to a consistent value. before reducing to a consistent value.
When the frame drop rate is dropped from 0.8 to 0.6, however, the connection When the frame drop rate is dropped from 0.8 to 0.6, however, the connection
becomes immediately unstable with bandwidth dropping to 0 Mbps as the connectio becomes immediately unstable with bandwidth dropping to 0 Mbps as the connectio
@ -7203,7 +7327,7 @@ LAN Premise Validation
The lag in latency response could be attributed to a couple of influencing The lag in latency response could be attributed to a couple of influencing
factors. factors.
The nature of a moving average partially representing previous values when The nature of a moving average partially representing previous values when
iterating adds a iterating adds
\emph on \emph on
friction friction
\emph default \emph default
@ -7217,7 +7341,7 @@ The initial spike in latency could be a result of the delay in drop rate
Additionally, the drop rate was employed at the client prior to queueing Additionally, the drop rate was employed at the client prior to queueing
frames in the transmission buffer. frames in the transmission buffer.
Frames are then delivered to the server decoupled from this rate. Frames are then delivered to the server decoupled from this rate.
As a result when a new drop rate is received the buffer is filled at this As a result, when a new drop rate is received the buffer is filled at this
rate going forward however the existing population queued at a different rate going forward however the existing population queued at a different
rate must still be transmitted. rate must still be transmitted.
\end_layout \end_layout
@ -7225,11 +7349,10 @@ The initial spike in latency could be a result of the delay in drop rate
\begin_layout Standard \begin_layout Standard
Following the implementation of a dynamic step, the latency was no longer Following the implementation of a dynamic step, the latency was no longer
unbounded and instead oscillated as the step increased and decreased. unbounded and instead oscillated as the step increased and decreased.
As a result the system can be seen to dynamically drop frames in order As a result, the system can be seen to dynamically drop frames in order
to maintain a constant latency in adverse network conditions. to maintain a constant latency in adverse network conditions.
This allows the system to keep up when when the amount of frames being This allows the system to keep up when the amount of frames being produced
produced by the client is too high to be transmitted in the same period by the client is too high to be transmitted in the same period of time.
of time.
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -7245,11 +7368,10 @@ The performance of this system, however, should be considered.
\begin_layout Standard \begin_layout Standard
One method to control this would be to have the step move in a non-linear One method to control this would be to have the step move in a non-linear
fashion, increasing at a higher rate the further away the live value is fashion, changing at a rate proportional to the difference between the
from the requirement and dropping faster as it falls below the required live value and the requirement.
value.
This could allow the latency to oscillate closer to the requirement with This could allow the latency to oscillate closer to the requirement with
the goal being for it to settle at that value. the goal being for it to settle at this value.
\end_layout \end_layout
\begin_layout Subsubsection \begin_layout Subsubsection
@ -7268,7 +7390,7 @@ With the employed method of controlling transmission latency, representative
\begin_layout Standard \begin_layout Standard
The need to deliver a step from the server to each client created an initial The need to deliver a step from the server to each client created an initial
spike in latency in both environments but was more pronounced in the cloud. spike in latency in both environments but was more pronounced in the cloud.
Additionally there were issues with the step not being delivered to the Additionally, there were issues with the step not being delivered to the
client at all, as this is included in the header of each request it was client at all, as this is included in the header of each request it was
concluded that requests were either being lost or queued at the server concluded that requests were either being lost or queued at the server
unable to be transmitted over the socket busy receiving frames. unable to be transmitted over the socket busy receiving frames.
@ -7337,8 +7459,8 @@ active
\noun on \noun on
Kinect Kinect
\noun default \noun default
sensors a method to tightly match different source frames within the processing sensors, a method to tightly match different source frames within the processin
pipeline based on timestamp could be implemented and tested. g pipeline based on timestamp could be implemented and tested.
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -7376,8 +7498,8 @@ ARFoundation
Metal Metal
\noun default \noun default
graphics library, limiting functionality to just that of the network layer. graphics library, limiting functionality to just that of the network layer.
With a compatible shader the cross-platform migration would be feature-complete With a compatible shader, the cross-platform migration would be feature-complet
, extending the possible install base of the application while also providing e, extending the possible install base of the application while also providing
access to the currently wider feature set of iOS's access to the currently wider feature set of iOS's
\noun on \noun on
ARKit ARKit
@ -7433,6 +7555,8 @@ frame types
will change significantly each frame and as such calculating and transmitting will change significantly each frame and as such calculating and transmitting
only the changes since the last could significantly reduce the required only the changes since the last could significantly reduce the required
bandwidth. bandwidth.
The challenge here would be establishing temporal coherence between successive
frames to identify these differences, a harder task in 3D.
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -7454,7 +7578,7 @@ name "sec:Summary"
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
Within this piece the process of extending the Within this piece, the process of extending the
\noun on \noun on
LiveScan3D LiveScan3D
\noun default \noun default
@ -7480,11 +7604,11 @@ The
\noun on \noun on
LiveScan3D LiveScan3D
\noun default \noun default
suite is described, both in it's initial state and following developments suite is described, both in its initial state and following developments
made to the network behaviour to include a sub-system of buffers. made to the network behaviour to include a sub-system of buffers.
Both accompanying AR applications for the suite are presented and finally Both accompanying AR applications for the suite are presented and finally
the suite as a whole is evaluated with it's strengths and limitations highlight the suite as a whole is evaluated with its strengths and limitations highlighte
ed. d.
\end_layout \end_layout
\begin_layout Standard \begin_layout Standard
@ -7616,6 +7740,30 @@ options "bibtotoc"
\begin_layout Section \begin_layout Section
\start_of_appendix \start_of_appendix
Project Gantt Chart
\begin_inset CommandInset label
LatexCommand label
name "sec:Gantt-Chart"
\end_inset
\end_layout
\begin_layout Standard
\noindent
\align center
\begin_inset Graphics
filename ../media/GanttChart.png
lyxscale 30
width 100col%
\end_inset
\end_layout
\begin_layout Section
Server UI Additions Server UI Additions
\begin_inset CommandInset label \begin_inset CommandInset label
LatexCommand label LatexCommand label

BIN
media/GanttChart.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 127 KiB

BIN
media/TestCloud.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 96 KiB

BIN
media/TestDev.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 42 KiB

BIN
media/TestLAN.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

BIN
resources/TestScenarios.odg Normal file

Binary file not shown.