One of my goals is to build a city builder or some kind of 3D game. So far I’ve done all but open up an editor and built something in webgl to get the screen cleared. In order to have something relatively comprehensive at a starting point, I have considered putting together a game engine myself, just to get my orientation together about what precisely I even need for something like this. Laying out the basics I might consider:

  • ECS system (this seems to be a common pattern used in game design)
  • Input (common inputs to control and interact)
  • Camera (what should the cinematography be like here)
  • Terrain
  • UI Controls

Part of the difficulty in starting anything is attempting to articulate precisely what it is you are trying to build. To build is to consider what you are attempting to manifest, what you imagine could be out of nothing. One of the essential ideas in putting something together is translating a vision - a visual conception of an idea into reality. That is to see it, and interact with it, to observe and define the behavior of the systems and subsystems therein. So, I think that the first step in exploring what it is I might want to come into being, is to start with defining the environment.

The crux of the situation however, is that there are constraints in how I might be able to build a sort of realistic environment without tremendous knowledge, utilizing an existing engine, or designing my own from the ground up. There is however, some alternatives in the space of terrain generation with recent work using Generative Adversarial Networks.

Interactive Example-Based Terrain Authoring with Conditional Generative Adversarial Networks:

` Despite more than thirty years of research in terrain modeling, au-thoring virtual terrains by using contemporary techniques remainsa demanding task. One reason for this difficulty is the wide variety of geomorphological processes that control the terrain shapeformation. Terrains are exposed for thousands of years to different erosion and land-forming agents such as water erosion, varyingtemperatures, vegetation, that are difficult to express by simple andversatile algorithms that could be used as editing tools. `

In fact, on top of this technique there is also the use generative adversarial networks used in compression techniques for physics in game engines, better rendering techniques and others. To get anywhere with this, the first step might be to simply read through the rest of that whitepaper as it pertains to the exact thing I would like to create.

The heart of the paper starts with the pre-processing step using a set of example data-sets to produce a set of units they call Terrain Sythesizers. The terrain sythesizers consist of four types:

- Sketch-to-terrain sythesizer **S**: creates a terrain from a sketch containing rivers, ridges, altitude cues, or combination of the tree
- Levelset-to-terrain sythesizer **L** turns a binary levelset image into a terrain
- Eraser sythesizer **R** removes a user-specified part of the terrain and completes it
- Erosion sythesizer **E** transforms an input terrain into the corresponding eroded terrain

Each sythesizer is specialized to a specific task, a corresponding dedicated database from real-world examples to learn each sythesis is necessary. The training, performed once and for all, is important since the quality and realism of the terrain produced in an authoring session is strongly correlated with the learned sythesis ability. Additional sythesizers could be added to the pipeline to consider other kinds of sketches or additional details.

The paper also references a technique for post-processing once a sketch and generated large-scale terrain has been created by supplementing it with small scale details using a super-resolution technique. The algorithm from Guerin et al. 2016 refers to terrain amplification as it considers information in the terrain in order to amplify it. It is well suited for the terrains generated using the cGANS method, because they contain coherent landform features but lack small scale details.

Following how terrain amplification and general authoring techniques work, the paper then moves on to describe how a cGAN actually works and functions. The cGAN consists of a pair of deep networks:

* a generative network *G* able to generate *B* from *A*
* a discriminative network *D* able to discriminate between *real* pairs (*A*, *B*) data from the training set and fake ones (*A*, *G(A)*) data generated by the network *G*.

The name adversarial training derives from the fact that G is trained to fool D and that D tries to avoid being fooled by G. While the authors are mostly interested in the generative network G, D is crucial to the learning stage because its discriminative power conditions the quality of the generator G.

The wealth of knowledge here was quite overwhelming to say the least. However, I did come across several YouTube channels and specific WebGL resources for simply putting together some concepts like heightmaps based on elevation data. In addition to this, I came across some threejs implementations that put together very basic concepts such as:

1. setup rendering platform (threejs)
2. create a terrain
3. load heightmap
4. texture terrain
5. add "collisions"
6. control behavior with navigation/keyboards
7. add shaders for fog/weather

This was a fairly low-level and basic timeline for implementing something, anything. Following this example, I came across this particular useful youtube channel ( The use of webgl shaders for foliage, fog, particles, wind, dust simulation, and more. These details, when implemented as shader animations, give quite a bit of life into the environment. It was helpful for me to see some of this, most especially in a specific technology implementation such as in WebGL.

I plan on utilizing these resources to take the next steps towards having something to demo.


  • ThreeJS Terrain:
  • Creating Terrain in ThreeJS:
  • MapGen:
  • WebGL Foliage, Fog, Particles, Animations:
  • Terrain Party:
  • Neural Networks and PCG:
  • Evolving Patch-based Terrains:
  • Interactive Terrain Authoring with Conditional Generative Adversarial Networks: