adding signature, selector content, fig sc

This commit is contained in:
andy 2022-08-29 19:21:17 +01:00
parent 30bb38fb69
commit 63c6bc4fd9
11 changed files with 29 additions and 8 deletions

View File

@ -13,8 +13,12 @@ Holoportation is an area of research exploring the ability to livestream 3D envi
The holograms are captured in the form of a __point cloud__, a cluster of coloured dots that, when presented correctly, can appear like the original object. The holograms are captured in the form of a __point cloud__, a cluster of coloured dots that, when presented correctly, can appear like the original object.
{{< figure src="/images/holo-avatar.jpg" >}}
My undergraduate dissertation documented extending the [__LiveScan3D__](https://github.com/MarekKowalski/LiveScan3D) holoportation platform to allow multi-source streaming. The existing capabilities allowed a single scene to be captured in real-time and streamed elsewhere for viewing, multi-source allows multiple independent scenes to be received and composited at the same time (a many-to-one system). My undergraduate dissertation documented extending the [__LiveScan3D__](https://github.com/MarekKowalski/LiveScan3D) holoportation platform to allow multi-source streaming. The existing capabilities allowed a single scene to be captured in real-time and streamed elsewhere for viewing, multi-source allows multiple independent scenes to be received and composited at the same time (a many-to-one system).
{{< youtube NP0aVjuk5fU >}}
[Read More](/holo) [Read More](/holo)
# [Mixonomer](/mixonomer) # [Mixonomer](/mixonomer)

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

BIN
content/holo/Structure.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 203 KiB

BIN
content/holo/ballcap.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 166 KiB

View File

@ -21,12 +21,20 @@ The app works by capturing what is called a [_point cloud_](https://en.wikipedia
3 Research 3 Research
4 Photoshoot 4 Photoshoot
{{< figure src="Structure.png" caption="Client-server structure of the environment" alt="client-server structure" >}}
# Multi-Source # Multi-Source
![pair of subjects facing each other](pair.jpg)
My undergraduate dissertation was tasked with extending the original software to allow _multi-source_ streaming. The current system could stream one scene to one server for viewing. This scene being captured, whether by one camera or from multiple angles, is called a _source_. Multi-source operation allows more than one scene to be composited and displayed at the server or a connected AR client. My undergraduate dissertation was tasked with extending the original software to allow _multi-source_ streaming. The current system could stream one scene to one server for viewing. This scene being captured, whether by one camera or from multiple angles, is called a _source_. Multi-source operation allows more than one scene to be composited and displayed at the server or a connected AR client.
The development works by including an ID to indicate what source a frame of footage represents. The development works by including an ID to indicate what source a frame of footage represents.
{{< youtube NP0aVjuk5fU >}}
###### A couple of recorded sources operating in the virtual space. A third live one is connected part way through
# Mobile AR # Mobile AR
The main use for a platform like LiveScan3D is augmented reality using a mobile phone or Microsoft Hololens. Although the point clouds are suitable for rendering in both an AR and VR environment, the resolution and nature of the captured perspective is suited well to recreation in AR. The main use for a platform like LiveScan3D is augmented reality using a mobile phone or Microsoft Hololens. Although the point clouds are suitable for rendering in both an AR and VR environment, the resolution and nature of the captured perspective is suited well to recreation in AR.
@ -37,12 +45,18 @@ Despite successfully migrating the app to use the ARFoundation library, the app
As a result, the app works fine from a network perspective, but there is a purple rendering error instead of a hologram. As a result, the app works fine from a network perspective, but there is a purple rendering error instead of a hologram.
{{< figure src="mobile-holo.png" caption="AR app working on iOS, a purple rendering error where the hologram should be" alt="mobile screenshot" >}}
# Research # Research
{{< figure src="ServerWindow.png" caption="Server window with additional statistics including bandwidth and latency as exponential moving average" alt="server window" >}}
As part of my ongoing work with the holoportation research group, I have also conducted experiments into the suites network behaviour. The original software was suited well to the lab environment that it was written for but there are a number of limitations that affects its performance over the open internet. For one, it uses TCP for its data transmission; streaming protocols usually dont use this because of the overhead it incurs among other reasons. As part of my ongoing work with the holoportation research group, I have also conducted experiments into the suites network behaviour. The original software was suited well to the lab environment that it was written for but there are a number of limitations that affects its performance over the open internet. For one, it uses TCP for its data transmission; streaming protocols usually dont use this because of the overhead it incurs among other reasons.
The work that I did used a collection of virtual machines hosted in various global locations as an environment to measure quality-of-service stats. Streams were set up over varying distances to see how it affected values like latency and throughput. This led to a channel management system being written that would manually control the throughput of frames in order to prioritise certain operating parameters. The system proved effective and further expansions are being looked into. The work that I did used a collection of virtual machines hosted in various global locations as an environment to measure quality-of-service stats. Streams were set up over varying distances to see how it affected values like latency and throughput. This led to a channel management system being written that would manually control the throughput of frames in order to prioritise certain operating parameters. The system proved effective and further expansions are being looked into.
# Photoshoot # Photoshoot
![ballcap](ballcap.jpg)
The system uses a [_point cloud_](https://en.wikipedia.org/wiki/Point_cloud) to capture and transmit 3D video. When zoomed in with a small point size, the medium looked really cool with the black virtual background, see here for more. The system uses a [_point cloud_](https://en.wikipedia.org/wiki/Point_cloud) to capture and transmit 3D video. When zoomed in with a small point size, the medium looked really cool with the black virtual background, see here for more.

BIN
content/holo/pair.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 243 KiB

View File

@ -38,8 +38,7 @@ I wanted to see what an app like this looks like, what it involves to build it.
In the process of working on this project, I learnt how to create web servers with __Pythons__ [__Flask__](https://flask.palletsprojects.com/en/1.1.x/) module, how to deploy them to a cloud environment and how to interact with other cloud-based services. The architecture is now completely serverless using __Googles App Engine__, __Cloud Functions__ and __Firestore__ services. In the process of working on this project, I learnt how to create web servers with __Pythons__ [__Flask__](https://flask.palletsprojects.com/en/1.1.x/) module, how to deploy them to a cloud environment and how to interact with other cloud-based services. The architecture is now completely serverless using __Googles App Engine__, __Cloud Functions__ and __Firestore__ services.
![cloud structure](cloud-structure-3.png) {{< figure src="cloud-structure-3.png" caption="Cloud architecture of services in Googles Cloud Platform" alt="cloud structure" >}}
###### Cloud architecture of services in Googles Cloud Platform
The front-end was written in __React__, which I also learnt in the process. It was, in fact, my first significant modern __Javascript__ project utilising a __Node__ + __Webpack__ stack, it was interesting getting to grips with the __Js__ ecosystem by making them work together and getting the result to deliver correctly from the backend. The front-end was written in __React__, which I also learnt in the process. It was, in fact, my first significant modern __Javascript__ project utilising a __Node__ + __Webpack__ stack, it was interesting getting to grips with the __Js__ ecosystem by making them work together and getting the result to deliver correctly from the backend.

View File

@ -10,9 +10,7 @@ I've been working on my .NET skills recently and, as I tend to, practiced with a
Selector is an agent that watches what youre listening to on Spotify and reacts to changes by firing pluggable events. These include retrieving the current songs characteristics and play count on Last.fm. This information is displayed on a dashboard that updates live. Selector is an agent that watches what youre listening to on Spotify and reacts to changes by firing pluggable events. These include retrieving the current songs characteristics and play count on Last.fm. This information is displayed on a dashboard that updates live.
![dashboard example](dashboard.png) {{< figure src="dashboard.png" caption="The dashboard shows information from Spotify and Last.fm" alt="dashboard example" >}}
###### The dashboard shows information from Spotify and Last.fm
The app consists of a ASP.NET web app and a command line service that runs the listening agents. A Redis instance is used for cache and messaging between the nodes. The app consists of a ASP.NET web app and a command line service that runs the listening agents. A Redis instance is used for cache and messaging between the nodes.

View File

@ -1,17 +1,23 @@
<footer class="footer"> <footer class="footer">
<section class="container"> <section class="container">
© <!-- ©
{{ if (and .Site.Params.since (lt .Site.Params.since now.Year)) }} {{ if (and .Site.Params.since (lt .Site.Params.since now.Year)) }}
{{ .Site.Params.since }} - {{ .Site.Params.since }} -
{{ end }} {{ end }}
{{ now.Year }}
{{ with .Site.Params.author }} {{ . }} {{ end }} {{ now.Year }} -->
<!-- {{ with .Site.Params.author }} {{ . }} {{ end }} -->
{{ if (and .Site.Params.license) }} {{ if (and .Site.Params.license) }}
{{ i18n "licensed_under" }} {{ .Site.Params.license | safeHTML }} {{ i18n "licensed_under" }} {{ .Site.Params.license | safeHTML }}
· ·
{{ end }} {{ end }}
{{ if (and .Site.Params.commit .GitInfo) }} {{ if (and .Site.Params.commit .GitInfo) }}
[<a href="{{ .Site.Params.commit }}/{{ .GitInfo.Hash }}">{{ .GitInfo.AbbreviatedHash }}</a>] [<a href="{{ .Site.Params.commit }}/{{ .GitInfo.Hash }}">{{ .GitInfo.AbbreviatedHash }}</a>]
{{ end }} {{ end }}
<img src="/images/andy.png" width="80px" />
</section> </section>
</footer> </footer>

View File

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

View File

Before

Width:  |  Height:  |  Size: 334 KiB

After

Width:  |  Height:  |  Size: 334 KiB