Date: 2021.12
Role: Sole Developer
As the sole developer, the project was developed entirely by myself from design to programming.
About
Project Genscapes is a proof-of-concept tech demo on AI-powered procedural landscape generation. Inspired by AI-driven content creation tools such as Runway and Artbreeder, the demo uses a custom neural network model to synthesize realistic-looking landscapes.
Serverless Deployment
Instead of running the model offline, I decided to deploy the model by using cloud service providers. I wrote Python code to receive HTTP requests, handle and process input data, run model inference, and send the results back to the client. I then packed the models and the code into a Docker image and deployed it using AWS Lambda.
Fully Responsive UI
Using anchors, pivots, layout groups, etc., I made the UI fully relative and responsive.
Building the Model
Using multiple publicly available geographic data sources, I created a custom heightmap dataset with more than 200,000 data points from the ground up.
To ensure the dataset’s quality, an automatic data filtering pipeline using noise estimation was built and used to remove blurry and noisy data.
To train the generative heightmap model using this custom dataset, I created and used a modified StyleGAN2 model to make it support 16-bit grayscale image training.
Latent Space Exploration
The demo features a variety of intuitive methods of exploring the latent space.
Random Generation
Pressing run with an empty grid executes random generation, filling the grid with random samples. This is the most basic and straightforward way of retrieving samples.
The truncation value parameter controls the quality and variety of the sampled heightmaps. Higher truncation values result in more diverse samples, while lower truncation values result in better quality samples.
Proximity Search
Similar samples with a specific heightmap can be found using proximity search. Loading a heightmap sample into the center item of the grid switches the mode to proximity search.
The search distance parameter determines how similar you want the samples to be to the loaded heightmap. Higher distances result in less similar heightmaps being sampled.
Grid Interpolation
Interpolate between four samples using grid-based interpolation. By loading heightmap samples into the four edges of the grid, the mode switches to grid interpolation. You don’t need to load heightmaps for all four edges to use this feature, as edges left blank will be automatically filled with a random sample.
Two-Point Interpolation
Blend between two heightmaps using two-point interpolation.
Visualization
The demo currently supports four different visualization presets. Press load and select the heightmap you wish to visualize.
Cancelled Features
Projection
Projection
Iterative projection of a given image to the latent space was implemented and tested at the research stage. However, it couldn’t make it to the tech demo due to server implementation issues.
CLIP Guidance
Result with the text "Desert Dunes"
Result with the text "Spikes"
Using CLIP and differential rendering, a feature for text-to-heightmap was implemented and tested at the research stage but couldn’t make it to the tech demo for the same reason as above.