coverpage
Title Page
Credits
About the Author
Acknowledgments
About the Reviewer
www.PacktPub.com
Why subscribe?
Customer Feedback
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
Lets Begin!
Welcome to the HoloLens!
The device
A wearable computer
The sensors
Cameras
Microphones
Holographic lenses
Spatial speakers
Processors
The device in a nutshell
Displaying 3D images
Wearing the device
Turning the device on for the first time
Learning the gestures
Calibrating the device
The device portal
Overview of the portal
The different menus in the portal
The Views menu in the portal
Writing your first holographic application
The tools we need to write software
Your very first application
Deploying your first app to the emulator
How to use the emulator
Deploying the app to the device
Deploying through Wi-Fi
Deploying through USB
Let's have a look at the code
The structure of the project
Inside the AppView class
Using Unity3D to create great holographic apps
Our first HoloApp in Unity3D
Fixing the camera
Changing the position
Changing the background color
Adding holographic objects
Create a sample script
Building the project
Continuing in Visual Studio after Unity
The structure of a Unity HoloLens UWP app solution
Build configurations with Unity
Building and running your HoloLens Unity app
Summary
Designing Your App
Unique features of the device
Thinking in three dimensions
Making sure objects are in a logical place
Making sure objects can be reached
Making sure objects can be found
Pointers
Sounds
Dealing with limited input methods
Gestures
Audio
People do not expect sudden changes
The social aspect in collaborative situations
Give your user superpowers!
Forming a design team
Big projects need big teams
Small projects need small teams
Using your experience
Where holographic apps are "just another app"
Where do holographic apps differ?
The workflow to identify your features
Writing down your mission statement
Brainstorming all ideas
Removing ideas that don't support the mission statement
Separating must-haves from nice-to-haves
Prioritizing must-haves
Implementing your app
Introducing the RockOn app
What do we want to achieve?
How do we get there?
Decision time - tooling
Developing with DirectX
Developing with Unity3D
Creating a new project
Getting started with Unity3D
The role of Visual Studio in developing for the HoloLens
Adding assets to the scene
Building the project
Deploying the project
Using the emulator
Using the device
Making some small adjustments
Summary
Gestures and Events
Working with gestures
Some UX considerations
A brief history of user interfaces
Command-line interface
Graphical user interface globally
Natural user interface
Feedback
Types of feedback in the HoloLens world
Getting to grips with gaze
Gaze in DirectX
The bare HoloCursor class
A word about shaders
The processing pipeline
Programming shaders
Adding shaders to our project
Additional data
Defining images in three dimensions.
Our square cursor
Rendering and updating
The Render() method
Updating after changes
Using our new class
Positioning
Rotating
Next steps
Gaze in Unity
The cursor
Creating the assets
Editing the script in Visual Studio
Using the script in Unity
Implementing the cursor code
Testing for touch
Events
Gestures in DirectX
The InteractionManager class
The SpatialGestureRecognizer
Gestures in Unity
Creating a prefab
Creating the creation code
Summary
Make Some Noise - Sounds
How our bodies work when it comes to sound
Head Related Transfer Function
Left and right
Front and back
Up and down
Head Related Transfer Functions
Moving our heads
Echo
Spatial sound in computer programs
Sound design
The design principles for spatial sound
The benefits of spatial sound
Grounding
User attention
Immersion
Interaction design
Best practices in sound design
Applying these principles
Licensing
Background music
DirectX
Getting the right NuGet packages
Loading and preparing the sound we want to play
Playing the sound
Actually play the sounds
Unity
Spatial sound
DirectX
The SpatialSound library
Using the library
Unity
Setting the audio settings in Unity3D
Adding spatial sound to GameObjects
Play the sound at the right time.
Some final thoughts on spatial sound
General concepts when it comes to sounds
What is next?
Summary
Be Heard - Voice Interactions
Let the HoloLens speak
General principles behind voice generation
Voice generation in DirectX
Creating the base of the VoicePlayer class
Initializing the sound system for streaming
Implementing IDispose the right way
Playing the sound
Synthesizing voice
Setting things up
Calling Play to hear the voice
Making the voice spatial
Voice generation in Unity3D
Creating the Unity app
Adding the helper script
Creating the scripts for the asset
Our SpatialVoicePlayer script
What is it about these #if statements?
Asynchronous stream generating
Creating the reusable asset
Adding voice to RockOn
Making things more lifelike
Introducing the SSML stream
Overview of the most important SSML tags
Listen up - voice commands
Default commands
Select
Hey Cortana
App commands
Hologram manipulations
Dictation
Troubleshooting speech recognition
Building a VoiceRecognizer in DirectX
The VoiceRecognizer class
Building the VoiceRecognizer in Unity3D
What is next?
Summary
Take a Look around You - Spatial Mapping
VR AR and MR
Virtual reality
Augmented reality
Mixed reality
Usages of spatial mapping
Occlusion
Navigation
Placement
Visualization
Physics
Masquerading imperfection
Things to be aware of in spatial mapping
Errors in the spatial mapping
Holes
Fake surfaces
Misaligned surfaces
Causes for errors
The user's movements
The physical qualities of the surfaces
Lighting
Moving objects
App-specific requirements
Optimizing meshes
Spatial understanding in DirectX
Initial app
Spatial capabilities
Initial classes
Checking capabilities
Setting up device-dependent resources
Creating the SpatialSurfaceObserver
Getting the surface data
Getting the actual mesh data
Pruning the list
The SurfaceMesh class
Updating and rendering
Mesh updating and rendering
Cleaning up
Putting it all to work
Spatial understanding in Unity3D
Writing scripts for spatial mapping in Unity3D
Adding surface mapping
Capabilities
Spatial mapping in code
Handling changes in the surfaces
New and updated surfaces
Wireframe
The easy way
Occlusion
Summary
Lets Talk! Communications between Devices
The position of holograms
Local coordinate systems
Spatial mapping and coordinate systems
Spatial anchors
Using the spatial anchors in DirectX
Using SpatialAnchors in Unity3D
Changing GestureHandler.cs
Problems with anchors
Stationary frame of reference
Using the stationary frame of reference in DirectX
Using the stationary frame of reference in Unity3D
Spatial locator-attached frame of reference
Using the locator attached frame in DirectX
What this does?
Using the locator-attached frame in Unity3D
Sharing
Setting up networking
Sharing scenarios
Using a hub
Setting up the data
Setting up the TCP/IP layer
Creating the thread for handling data traffic
Handling connections
Sending data
Creating the mixed-reality client app
Setting up the connection
Consuming the connection
Connecting from Unity3D
Peer to peer
Introducing Bluetooth advertisements
Setting up the advertisement
Watching for other devices
Handling found devices
Connecting to the other HoloLens
Sharing world coordinates
Sharing spatial anchors in DirectX
Should you send one or multiple anchors?
Sharing world anchors in Unity3D
Sending the data from Unity3D
Receiving the data in Unity3D
Summary
Speed Up Your Development - Advanced Unity3D
Framework versions
Mono runtime
.NET in HoloLens
Script compiling in Unity3D
Compilation errors and how to fix them
Three possible solutions
Option 1 - do not go back and forth
Option 2 - sprinkle the code with #if/#endif statements
Option 3 - placeholder assemblies
Why does this work?
Performance in Unity3D
Remoting in Unity3D
Profiling in Unity3D
Unity3D performance tricks
QualitySettings
Polygon/vertex reduction
Static objects
Level of Detail
Lighting optimizations
Texture optimizations
Scripts optimizations
Threading
Garbage collections
Structs versus classes
Shader optimizations
Introducing the HoloToolkit
Where to get it
Importing the toolkit
Using the toolkit
Menu items
Assets
Shaders
Sharing service
And much more
Summary
更新时间：2021-07-02 19:09:04
完结共353章
倒序