Notifications
Article
Toward Procedural Generation
Updated 17 minutes ago
102
0
A brief article about preliminary work concerning the procedural generation of diverse environments for complex autonomous (mobility) systems
[DRAFT 0.0.1] Still quite a bit of TODOs ...
This article is the first part of a four part series which shall give a (brief) introduction to preliminary methods pertaining to the development and simulation of complex autonomous systems. This first post is to introduce some bits of the "lower level" points of the platform. The next post (Toward Autonomous Agent Simulation) shall present a view of the simulation ecosystem from a higher level, and further explain more details from a more "outside" perspective. The final two posts (Toward the Simulation of Vehicular Flocks (Crowds) and Toward Decentralized Complex Autonomous Organizations) are intended to demonstrate a currently "in development" projects / applications that shall use the AgentSim Ecosystem (the platform).

Preliminary Notes

[TODO: Convert all links...]
The Perspective: Ontology, Ontological Engineering (see [48], ch. 12), Simulation, Procedural Generation, Serious Games, Simulation of Serious Games (SSG),
*This is all very much an "in development" work, and as such may one find various "incompleteness-es", notes, general thoughts (and perhaps many questions), and maybe even (but hopefully minimal) errors by the author spread throughout this document (and all connecting documents). Shall the final paper of this work be a more complete, rigorous, concise, unambiguous and correct (as the work can possibly be made to be) product. This work, as is said about all of these early works on this field of study, is a "living draft", and is therefore not yet a "formal treatment" of the subject matter, but the (incipient) means to eventually arrive at such a final, formalized product.
*A paper on this work of procedural generation is also currently in the works. This is to say that this article shall serve as more of an introduction to some of the most important concepts surrounding the procedural generation methodology, and the paper shall give a significantly more rigorous exposition of the method.
*To add to the previous note, formal research publications are very much planned for the future, but as the project is in a state of such immaturity, does it not currently seem reasonable / feasible to publish any of this preliminary work in any capacity (other than in the informal blog format, of course). Any formalized publication is intended to be reflections of the final products of this (and future) work, and the current blog publication format and Github repositories are demonstrations of intermediate, "in development" processes as which to achieve the final products and objectives (also, does this mechanisms serve as a means to keep a log / account of how the final product came to be (via the processes) [for study and analysis purposes]).
*"key words" sections have two purposes here: 1. to provide a point of reference of relevant topics for further investigation that contribute to the goals wanting to be achieved by this work 2.
*The exact definition and nature of "complex autonomous mobility systems" is explicated in more depth in [1] (though there does not exist a "formal" definition as of this moment (future work to come).
*Duplication article on Medium: https://medium.com/@domnic_eaton/toward-procedural-generation-553bb3d92fa6?sk=f67a2b7c7e9743a080db04d23e58e911 [TODO: WRITE / COMPLETE MEDIUM ARTICLE]
*A precedent set in previous works (see [1]), does the asterisk (*) symbol denote side notes or special (separate) points of interest or consideration.
*A follow-up article is to come which goes another brief introduction into the AgentSim simulation component platform, and further explains the role of this work on procedural generation has in the larger context of autonomous agent simulation.
*Is the use of the word "components" used rather heavily in this work, and as shall be mentioned later in the article, is a goal here to develop a system (pipeline, algorithm) that has a high degree of modularity, extensibility and interface-ability (and in general compose-ability ), and the fundamental unit of structure that creates the system is captured by the word "component". (One could suppose that this is a tendency toward a "mechanistic" and " reductionist" perspective of design)
*It should be noted that this article is purely introductory, and a significant amount of detail is omitted in a (somewhat difficult) attempt to maintain a sense of brevity and clarity of higher level, overall ideas.
*procedural generation as a means toward automation and as a method to create some semblance of a "mind".

A Side Step

To give a complex autonomous agent the tools for self expression is a interesting idea for which this work aims at studying in more detail. Not unlike how humans may have difficulty at times produced the correct words in giving speech and use other mediums (art, writing, etc...) for expression of their internal simulation, can an automated tool-set provide such a similar functionality to complex autonomous systems (agents) who desire (need, for intepretability) to express complex "thoughts" for various purposes. Furthermore, providing such systems with a "sandbox" as to test and develop their own internal states may be of great utility, as agents may have a "safe space" to perform internal simulations, which can be interpreted by humans (as the tools that they use were developed and are in turn interpret-able by humans), may grant agents the ability to perform higher level cognitive functions while gaining (simulated [though "real" and "simulated" experience are very much dependent on interpretation and perspective] [observer and perception biases]) "experience" at the same time.The automated creation of realistic worlds could be of great use in the development of these systems for the reasons outlined here and other (yet to be determined) reasons as well. Does this work not assume that the methods presented here shall be the only (or even optimal) ways at achieving such goals of agent "simulability" (simulate-ability), but does it at least hope the the question of whether or not a "realistic" simulation can be made to perform such capabilities is, at the least, possible.
To give further context, and relation to the application to flocks of autonomous [ground] vehicles in constrained roadway environments, one can imagine that a "flock" of such vehicles are attempting to navigate their highly constrained environment. "Think globally act locally" comes to mind here, as agents must "think" about global objectives, while acting / using / affecting only their local systems and environments. Agents have a need to "think" beyond themselves and "imagine" how their local interactions could have global consequences that affect other agents (theory of mind plays a role here as well), and clearly, are all of the possible achievable states are not easily (if at all) accessible to any given agent of computational bound. Simulation for humans would mean that a human can "peer" into the "mind" of these agents (so a human would generate a world [or let the agent generate its own interpretation of the world, with constraints, given its observation / perception of it] and can study and analyze these results.The agent, under study and observation, can perform a similar a task and critique, modify, and learn from itself (perform the act [functionality] of "autodidacticism") and can play out different scenarios to obtain an optimal solution. Simulation and automated generation of realistic and complex environments may yet serve as a connection point for human and machine (agent ) interaction, where both can analyze and modify the simulated environment and then translate these actions and results into the physical world for further testing and development. The physical world is constrained by reality, yet a simulated world need not be (for better or for worse) and as such is there an emphasis on the importance and necessary standardized integration of simulation [and automated generation] into the development of complex autonomous systems (of mobility [services]).
Simulation (and procedural generation) may benefit both the agents and those who use these agents (as a service), and for this reason is great emphasis placed on ensuring the understanding of the context of this work.
*Note, from a correspondence:
  • Remark: "...one thing that comes to mind ... is the arrangement of the human (and other creature) cognitive subsystems into a working whole. One can view it as a master computer networked to simpler, localized computers. The localized computers handle localized tasks effectively, and the master computer is in charge of “the big picture,” instructing the localized computers in things related to the community, as a whole."
  • Response " ...[the] point on this "master computer network" is one of the principle points of one of the applications that I mentioned that the simulation platform is being developed for studying, namely, Toward the Simulation of Vehicular Flocks (Crowds). The purpose of that work (toward vehicular crowds) is to study and develop mechanisms and methodologies for emergent behavior among autonomous road vehicles (in highly constrained environment [a distinction from our previous conversation actually]), yes local subsystems do form together and become part of a larger "complex autonomous mobility system" (the terminology that I use to describe these networks of "self-driving cars"), but on of the deeper goals is to attempt to and "eliminate" the need for a "[central] master controller" that directs smaller, localized agents (each of whom follow relatively simple rules), and use properties related to emergence and collective intelligence (for example as can be seen in bird flocks [or any animal grouping mechanisms]), to create globalized, and more importantly, decentralized behaviors and solutions (in reference to mitigating traffic related issues, traffic congestion, traffic controllability, safety concerns, etc...). From there, can one imagine the creation of complex autonomous overlay networks (The Decentralized [complex] Autonomous Organization (DAO / DCAO) for instance), which can further develop these simple, localized agents, into a larger systems of increasing complexity that can concurrently solve increasingly complex issues (which is the topic of the other application mentioned in the article: Toward Decentralized Complex Autonomous Organizations) (in our previous conversation, I mentioned the creation of "flocking" agents to create "mobile [decentralized] markets", and in general do I believe that many other applications can stem from this idea as well [The Complex Autonomous City, automated services and applications of mobility, etc...]). This "decentralization" concept stems from work and ideas surrounding the "blockchain" technology and the concepts of "liquid democracies" where the collective of individuals form a unitary will (so the collective is self-directing and guiding, is there is not necessarily a single "leader"), is this concept is what is really desired to be explored here, as to see whether or not such concepts shall be applicable to, or of any utility to the development of autonomous vehicles for the future. So though I have only briefly mentioned here some of the applications for the procedural generation platform in the first place, the articles that I mentioned that further explicates these topics shall likely be of equal or greater length than the draft article that I have presented to you (just to note), and shall go into significantly more exposition of the subtitles, methods and connections among these topics that I have, so far, only "loosely" described."
*Note: A similar argument is expressed in the side notes of the article "Correspondences: on Comparison and Consistency" (see [47]) where the idea of "symmetries" (transform in-variance) is thought to be a key point f interest in producing such behavior of creating real worlds that need not be exact copies of any given (physical) world, but each world abides common laws and principles shared between them (consistency) yet still introduces sufficient diversity and variety as to sufficiently "challenge" a give complex autonomous [mobility] system in development (or under study).

Introduction

Here, shall I give a brief introduction to some initial work being done in terms of procedural generation of environments. The goal is merely to introduce the work and not give too much of a detailed account of the entire project, which shall be done at a later time. Today, may one find many works aimed at recreating the "real world" as a way to allow for the development and testing of complex autonomous system (autonomous vehicle for example [complex autonomous mobility agent systems]). As with any problem, however, there exist more than one ideal solution. Toward the problem of generating a sufficient amount of suitable data of variety and diversity, of which has the purposes of development and testing of autonomous system related technologies, perhaps the approach of automation (procedural generation) may also be of great utility. Much work is also being done in the field of procedural generation (see [3-15]), and so a point of this work is to not necessarily create a completely novel approach to procedural generation of complex environments, but to just create a significantly more comprehensive one.
Given a standardized method for procedural generation, is there the hope to then create a formalized simulation system (a formal methodology) for the simulation of complex autonomous (mobility) systems. In general, does this work have the goal of (from the high level perspective) of finding suitable control mechanisms for such complex autonomous systems. In any given control problem, does one first need to perform some type of system identification (finding suitable system representation [model]), before a suitable control law can be created. It is a desire that the procedural generation methodology of this work shall be a solid first step forward in the direction of designing, developing, and testing appropriate control mechanisms for any given complex autonomous system.
Quick Overview
This work then aims at devising a procedural generation tool that can create environments, starting from a truly empty space (a four dimensional Haussdorff 4-dimensional manifold [3-dimensions of space and 1 dimension of time [Minkowski spacetime] representing the spacetime continuum) , from which a procedural world is then built. This procedurally generated world then allows for the generation of increasingly more complex static structures (roads, parcels, buildings, settlements, etc...). Given this procedurally generated static world, is a dynamic world filled with "intelligent" agents then generated (pedestrians, vehicular traffic, etc...). These agents, in one sense, can then present a challenge to some complex agent (or network of agents) under study (an autonomous ego vehicle for example, or a flock of such vehicles). This methodology of procedural generation presented in this work is hoped to completely automate the process of creating all necessary components involved in developing and testing autonomous system technology, where a user would only need to change some parameters, options, or settings list or file, and the user would not need to concern themselves with the many details involved in creating such environments of satisfactory complexity and variety.
This work of procedural generation and automation of environment and design and testing scenarios is, in fact, the first part of a larger body of work (AgentSim, see [1 ]), where this larger work hopes to create a more complete and comprehensive tool, environment, and ecosystem for complex autonomous system simulations. This work, and the larger work, are still both very much in their infancy, yet it is hoped that the final tool will be a first step toward streamlining the development of complex autonomous systems and allow for more "standardized" approached to developing these systems which also, ideally, allow these systems to be created with great ease.

Background

I shall only put high level references and basic information here, as each of the topics has vasts amounts of information associated with them, and it is desired currently to just briefly demonstrate the roles that each subject has to play in this work. As such for now, more information about each subject can be found in the References section.

Space, Time and Change

There is a significant amount of information to be studied and discussed here (which I shall do at a later period of time [likely in the technical paper outlining the work of procedural generation]), and as such, out of pure desire to not fall into a "black hole of information/knowledge" (a singularity point of infinite space and time) and so shall I just give a high level allusion to some reasons for even looking into this field for consideration.
The method of procedural generation that is hoped to be created here is one such that shall be of great comprehensiveness and shall provide an accurate model of the physical reality and all things contained within. Though the exact nature of reality is still not necessarily completely understood, it may still be worth the effort to at least take some of the most fundamental conclusions from the field of (theoretical) physics and apply them in our method of procedural generation of realistic environments. The main purpose here for even having such considerations is more so as to form a theoretical foundation for formally devising a method for procedural generation and suitable representation (through rigorous mathematics and physics).
The subjects of Topology, Differential Geometry, Geodesics, Spacetime, and Chaos Theory are all points of study and interest for this work, as it is believed that crucial insights about foundational and rigorous representations and manipulations of topological / geometric structures can be gained and that could allow for a formal exposition of the procedural generation process (from both a static and dynamic perspective).
Topology
A rather large subject that concerns itself with techniques related to geometry from a perspective of continuity (in conjunction with differential geometry), does this work aim at applying the conventions of this field (and that of differential geometry) in a manner as to formally define the method of procedural generation of (topological / geometric) environments. Here are topics such as metric spaces (an intuitive / generalized notion of distance in a given space, and is a central concept in both topology and differential geometry) , homotopy (shape / path deformation characteristics), ..., and several other topics become relevant. Through the consideration of various ideas of topology, is it hoped that basic intuition about how one could begin to compose and morph shapes in a given space (manifold) given some concept of distance between these shapes (the metric space) can become possible. (see [24])
key words: metric spaces, topological spaces, compactness, homology theory, cohomology, homotopy, homeomorphism, Hausdorff space, sequences, nets (generalization of sequences), spatial continuity, spatial transformation in-variance, frame of references
Differential Geometry
Extending a bit beyond vanilla topology, does the study of Differential Geometry lead to notions about the nature of curves and surfaces (see []). Space (from spacetime), the planets contained within the space, and the environments that exist within the planets are all "curved", and do methods and study of differential geometry naturally apply to geometric objects in this setting.
key words: manifolds, geodesics, cartography, atlas, charts
Spacetime
Is it really not a goal (at least for now) to include or use the deep intuitions about the current understandings surrounding the theory of relativity (or special relativity), but really just to use some of the basic (mathematical) machinery and notations for modelling and representing various geometric and topological objects with respect to space and time.
(M, g) M - spacetime manifold g - metric tensor (rule of distance)
key words: spacetime
[TODO: complete section...]
Chaos Theory
Dynamic systems who are capable of highly random behavior (due to sensitivity of initial conditions) are of focus in the theory of chaos.
(M, f) M - spacetime manifold f - evolution (rule) of state
key words: inertial frames
[TODO: complete section...]

Tensors

Tensors are of particular interest for many reason (not just for the procedural generation of road networks) as they provide a concise representation of physical state in mathematical terms. So though in this work, only information relevant to road generation is presented, in reality the use and study of tensors shall likely be seen in many other areas as well (such as the representation of the complex system of study itself (see [9])).
Representations and Bases
Much of the work toward the procedural generation of roads is based from the work of [14], and its derivative works.
*Note: Is there also the intention to include functionality to import maps from OpenStreetMaps - https://www.openstreetmap.org/#map=4/38.01/-95.84
[TODO: complete section...]
keywords: Hyper-streamlines, Degenerate Points (road network intersection points),

Computational Geometry

Computational Geometry provides a series of algorithms and methods for performing the algorithmic manipulation of geometric elements, which is especially useful in producing and manipulating meshes in 3D computer graphics.
Triangulation
The creation of a polygonal "mesh" in a virtual 3D environment requires the triangulation of such an object. Triangulation (with respect to this work) essentially involves taking a series of points which represents some geometry 3D (or 2D) object and finding an optimal set of triangles (which is what Delauney triangulation method attempts to do [there exist other methods for triangulation as well]) that can cover the surface of the object. Given an object with an optimized set of triangulation, can a renderer produce a "mesh" in the virtual environment. This figure below is just a random example of the Delauney triangulation method, but does Figure [] show an example triangulated road mesh (which can be rendered in Unreal 4 [and eventually in Unity / Blender...], via the code in the Github repository: "TransitCreator-AVSIM_Module" ),
key words: ear clipping,
Convex Hull
The idea of convex hulls is to find a the outermost boundaries of a set of points (which can clearly be seen by the red lines in the figure below, given a set of blue points). This algorithm can alle be used for 3-Dimensional points, and so an idea here is to be able to find a set of points that represent the outer (closed) bounds (so the outer surface, in a 3D object for example) of a geometric object, and then produce a triangulation of this which shall allow for the procedural generation of the object in a virtual environment. Introducing convex hull methods is more for allowing the creation of more arbitrary geometric objects (such as geometric objects produced from a random (coherent) noise function).

Interpolation
Interpolation essentially amounts to methods that seek to "fill in gaps" between a set of points. Interpolation can be extended to arbitrary dimensions and has many uses throughout this work (such as in the generation of Simplex Noise, or the approximation of arbitrarily curved surfaces). Similarly to the reasons for introducing convex hull methods, is a reason for having a generalized interpolation method such that approximations of arbitrarily curves or edges (for the examples some procedurally generated curved edge of a piece of a building [or road], created from coherent noise) can be found.


Polygon Partitioning
[TODO: complete section...]

Numerical Methods

Numeric methods involve "discretized" algorithms for approximated various concepts of calculus, such as integration. The calculation of hyper-streamlines of tensor-fields requires integration along a path of a given field, and in this work (see []), the RK4 method of integration is used for this.
key words: Runge-Kutta Numerical Integration
[TODO: complete section...]

Road Geometry Design

In an attempt to keep the realistic element to the procedural road generation algorithm, an idea is to generate roads based actual, real world, computational methods used to create the geometry of real roads. One can find the details (the equations and explanations of creating road and highway geometry) in the references [ ], but here do I just want to give some brief details on how more realistically procedural generated roads could be created.
The computation of the hyper-streamlines of some given tensor-field basis produces "station points" (centering points used to determine road alignment), and the idea here is, after this process, to produce approximate road geometry based on the equations used in road and highway construction engineering, some of which are described below.
[TODO: complete section...]
*to follow OpenDrive specification: http://www.opendrive.org/
key words: Horizontal Alignment, Vertical Alignment, Super-elevation, Cross-Sectional Alignment

Procedural Generation Techniques

The procedural generation of 3-Dimensional virtual environments is certainly not a new concept, and as has been stated before, much work has been done already in various areas of this field. As such, shall only give a brief overview of the most immediately relevant concepts to this work be stated here, and more information can be found in the references [2-12].
L-Systems
Lindenmayer Systems...
[TODO: complete section...]
key words: Parceling, Shape Grammars, Quad Trees, Cellular Automata

Coherent Noise

An important aspect of procedural generation is the concept of "coherence " (and consistency), where elements produced by the procedural generation methodology must have the characteristic of diversity, but must not have to great of deviation from physical reality. A sense of "connectedness" must exist between created elements in the pipeline (an interesting and possibly related concept may be that of "percolation" in statistical mechanics), yet also must there be a stochastic element of chance to achieve the desired variability. Naturally, does one come to consider the theory of coherent noise as a framework for producing such elements. The work of [16] began toward a more formal treatment of coherent noise, and it is hoped to extend these concepts into the larger context of this work.
[TODO: complete section...]
key words: Perlin Noise, Simplex Noise, Fractal Noise

*Stochastics

Though it is not explicitly mentioned in this work, probability, statistics, and in general stochastic modelling are a very integral part to the creation of highly diverse (and probabilistic) environments. Shall not anything more beyond this be said here, but does a point wish to be made that the field surrounding probabilistic modelling and its various techniques and methods are of great consideration and importance in the process of procedural generation.
Also, to just briefly mention, does another idea (currently being implemented by others) is to use "black box" models (and here do I mean data-driven models) for the creation of various procedural generation components (terrain, etc...), using captured real world (geographic) data (from LiDAR, Radar, etc...). Does this work take no issue with the use of such methods (versus using more formalized approaches [here do I mainly refer to the use of a formal theory of coherent noise / stochastics / statistics]), and more so does it begin to ask the question (beg the question, I suppose) of which method is appropriate when , or if a hybrid method is most suited for the procedural generation of complex environments (a question to be answered by future works). Generally, it is hoped that this framework can have the flexibility, agility, and adaptability to support either (or any) method for procedural generation (of sub-component parts).
key words: cumulants, moments, central moments, moment generating functions, probability distributions, conditional probability, qeueing theory, renewal process, Brownian motion, Poisson process, Markov chains / processes, discrete event simulation, Statistical mechanics, [statistical] Thermodynamics, Percolation Theory

Procedural AI Concepts

Here is just a highlight of some of the concepts and ideas used in the creation of the procedural artificial intelligence module. This work currently presents a more theoretic framework for the procedural generation of AI agents, and future work includes the development of a more standard architecture and pipeline for the generation of procedural AI.
[TODO: complete section...]
key words: Hierarchical Tree Planners (HTP), Blackboards, Behavior Trees, Influence Maps, Utility Theory, Observer Design Patterns, Publish-Subscriber Design Patterns

*Category Theory

It has been stated before that one of the objectives for his work is to develop a rigorous and more formal treatment of procedural generation methodologies, and from a mathematical perspective ifs there an idea to use the framework established by category theory to define these techniques (of procedural generation).
[TODO: complete section...]

Optimization

[TODO: complete section...]
key words: convex / non-convex optimization, multi-objective optimization, evolutionary algorithms, ...

Methodology

The overview section gave a high level description of the basic functionality of the procedural generation pipeline. As the currently, the work on the pipeline is still not yet complete, shall I only give some brief highlights on current work, and expand a bit further about some of the details of the pipeline.
The procedural generation methodology used to create diverse environments can be subdivided into three main components: the PWG, PAI and PCG components. The static elements and morphology (shape) of the environment are produced by the PCG module. Dynamic elements (pedestrians, and other moving agents of "intelligence") are produced by the PAI module. Detailed elements (textures, animations, etc...) are then finally produced by the PCG module. This pipeline (algorithm) is implemented through the GE component of the larger AgentSim Simulation component (all of which is further encompassed within the larger AgentSim Ecosystem). Consummate automation and streamlining the creation of complex and diverse environments is hoped to be achieved through the development of this procedural generation method, which shall further be used for the development of complex autonomous (mobility) systems. Though this work emphasis more of the system design for the procedural generation methodology, does the paper (to come) and the final work aim at providing a complete and standardized procedure for procedurally generated any environment (this has been stated before, but does emphasis want to be given to how these ideas all come together to form an automation continuum [complete automation tool]).
key words: PWG-Procedural World Generation, PAI-Procedural Artificial Generation, PCG-Procedural Character (Agent) Generation

The Pipeline

An initial outline of the algorithm [pipeline] is presented in the figure below. The is the first part of the total pipeline, that represents just the generation of static elements in a given environment.

Procedural World Generation (PWG)

The pipeline begins with an empty space-time manifold object (the entire process of procedural generation occurs with respect to space and time [spacetime]), and then the algorithm begins to populate this object with static elements, the first of these being worlds, terrestrial environments, and terrains. Terrain morphology (and the later development of Settlement formation) are to be principally generated according to user input (information maps (semantic maps, see [2]), options, settings, input parameters [in the form of json / yml files most likely])
Given a procedurally generated terrain, and using the information maps (generated from the user) are tensor (scalar --> vector) fields generated on the terrain, and from these fields are settlements generated. Settlements are divided into two main components, road networks and buildings, where the pattern / placement of these elements are based directly on the generated tensor field. Tensor-field generation and hype-streamline computations share parallels with the ideas of real world road geometry development, namely, land surveying and stationing. The hyper-streamline points represent the center-lines of the roads in the road network and, through use of the road geometry equations, are the road geometries approximated and generated and are placed into the terrain.
Currently, are the tensor-fields being generated through first deriving the characteristic vector fields (eigen decomposition) and then generated then two vector fields and superimposing them on one another (this is what is shown in the following two figures below).
In the work of [14], is there described a method for blending together different street patterns, in order to form a single cohesive road network. A road network then has the characteristic of composability, and more complex road network structures can be generated from the tensor-field bases (grammar).
Upon the generation of the road networks, can plot generation (the space between the roads in the road network) and parcels (the subdivision of plots for which buildings shall be generated upon) can be done. Inside of each parcel building structures (as generated according to the zoning (smart code) information map) can be created. Building footprints are generated as to create the foundation of which the building is built on top (plants and other ornaments are to be generated in a separate, concurrent thread). Given a building footprint and foundation, is a building to be generated as according to some shape grammars (the [probabilistic] rules of composition and the geometric objects used to construct a building). Building interiors are not of particular concern in this work at the current moment, but may become of more interest later (if needed). At the moment, are buildings just to be considered "rabbit holes", where dynamic elements (for example, procedurally generated vehicles [or pedestrians]) can freely enter or leave a building's exits / entrances. So these buildings operate as spawn points for procedural dynamic elements. An idea here is, for example, given a test scenario that involves an ego autonomous vehicle, the ego vehicle can have a certain (user specified) "radius of activation", where this activation radius (or ring) can activate the spawn points located within buildings (or other civil structures). So, when an ego vehicle enters into a new region. the spawn points activate, and procedurally generated dynamic elements begin to leave and enter the building structure (the behavior of the dynamic elements can also be specified by user defined information maps and models).

Procedural Artificial Intelligence (PAI)

Beyond just static elements, does this work also aim at creating a generalized framework for the procedural generation of "intelligent" dynamic agents. These agents are composed of components and (data /event driven) models. The models, which represent various facets of the agents (behavior, decision making, senses, etc...) provide the template for creating an given agent. From these model template are components made, and a hierarchical chain of components eventually build up into a complete procedural generated "intelligent" agent.
A goal here is to create a system that can produce a series of intelligent agents that can provide a sufficient challenge for a given autonomous agent (or network of agents) under study and development (Quite similar to the use of artificial intelligent agents in video games that are to challenge a player [in this case a "player" is an autonomous system (vehicle)]). Beyond the ability to provide a "challenge" to a complex autonomous agent, is there concurrently the goal to develop a framework that shall facilitate the study and development of networks of complex autonomous systems. A distinction is to be made here between the PAI component and the AI Platform component in the architecture of the Simulation component of AgentSim. the PAI has the purpose of accomplishing the first goal mentioned here (provide a challenge to autonomous system), and the AI Platform Component aims at accomplishing the second goal (studying, developing and testing networks of complex autonomous systems / agents).
Procedural intelligent agents are divided into two main internal components and an external component. The internal components include the Body component and the Brain (Mind / Controller) components, and the external component includes the AI Coordinator component (which includes the Blackboard, Influence Map and Pub-Sub coordination (Publisher-Subscriber [and Observer [sub]components]) sub-components). The two diagrams of the procedural intelligent agents demonstrate the relation between the components necessary for creating such agents, and the implementation inside the simulation component (simulation environment). Agents are defined through models (behavioral, data, cognition, dynamics, etc...) and their [sub] components, and are implemented (made to interact with their environment) through the Controller-System-Coordinator relation component (a better name still not yet to be found...).
*Note: The distinctions between the procedural intelligent agent and the autonomous agent of study are still not fully formed (the architectures may seem to blend together, many related concepts and terms may blur the lines between them, etc...) but it must be pointed out that there is a distinction between these two entities, and their implementations / methodologies, though they may share similar characteristics, are not intended to be the same. (If anything shall the procedural AI be a lightweight / "cheap" version of the autonomous agent of study (the ego vehicle for example)).

Procedural Character Generation (PCG)

Eventually, shall it be ideal to have a system that streamlines the process of creating dynamic intelligent agents (so procedurally creating characters with automatic rigs, animations, and detailed textures), and all of this functionality is to be included in the PCG module

AgentSim Simulation and "GE" Components

As is to be further explained in the follow up article to this one, is one of the goals of the simulation platform to have high modularity, extensibility and interface-ability with existing software (as was stated in the preliminary notes). With this in mind, may one find some particular patterns in the system design process (this is really just a side note to raise awareness of this goal of simplistic composition to engender complexity).In any case, in the diagram depicted below, one can see an expanded view of the procedural (static element) generation pipeline. It should be noted that , even though at the beginning of the Methodology section a linear algorithm was presented, is it likely thought that a multi-threaded procedural generation approach shall be used in the final implementation (for speed and computational purposes), which is also reflected in the diagram below.
The unfortunate shape of the diagram above makes the ability to clearly display the figure particularly difficult ... [TODO: fix this...]
To give a bit of context (which shall be explained in more detail in future articles) diagrams depicted the larger AgentSim Ecosystem and Simulation component are depicted below. The AgentSim Ecosystem has the purpose of providing a tool for creating, developing and testing various complex autonomous (mobility) agent components, which mainly include devices (and systems), controllers, and simulation related environments. This work focuses on the creation of a "master" simulation component (but in reality is the idea allow for the creation of a series of simulators [which is the same for the device and controller components]).
Going down the ecosystem taxonomy (abstraction) chain, does one arrive the "master" simulation component. Depicted in the diagram one, does one see the GE module component (the focus of this work), whose functionality includes all of the 3D rendering, physics simulation, and general visualization modelling. Each software module component (Core, Device Platform, GE, etc...) has the same basic structure (depicted on the right sided diagram), which includes a core software package (generally written in c++ (and tested with java / python)). The modules connect and communication with one another through upper and lower interfaces, and, in general, is there an API the provides a standard set of functions and method of interaction between core software packages and external software packages (external software packages of the GE component includes [or will include] code implementations in Unity, Unreal 4, and Blender Game Engine [BGE] that are wrapped inside of internal software containers) [*Note: this same pattern of design applies to all of the software component modules]. External software package containers are further subdivided into applications and libraries (for example, the procedural generation external software package includes the applications such as PWG, PAI, and PCG and libraries such as MathLib (really, the core mathematics library for the entire simulation platform) and PhysicsLib [and many others as well]).
As has been stated before, the procedural generation pipeline is implemented through the GE (Game Engine) component. The heart of the GE component is made from a combination of a scenario component and a management component. These two components work in conjunction to produce a scene (level, scenario, environment) full of static and dynamic elements. These static and dynamic elements are produced using the plugin modules, where each plugin module provides a different aspect of the procedural generation pipeline.
At the top of the module component stack is the "Applications" module component, for which a synthesize of the previous component modules forms the basis for creating applications and services (or experiments and scenarios) of any appropriate system). At the very top of the simulator component of the ecosystem, sits the AgentSim user interface. This is the main interface for which users are intended to interact with the entire simulator, and has a goal of hiding the (somewhat unavoidable, I fear) complexity of functionality the simulator component has to offer.As is stated in the conclusion, the AgentSim ecosystem is hoped to be "software agnostic", meaning that it has no "hard" dependencies on any of the sub-software[s] of which it is comprised, but rather does the ecosystem provide a standard set of algorithms, functions, and methodologies, all of which are readily accessible and utilized through the high level Agentsim User Interface. The AgentSim user interface is intended to be the central point of access to these various sub-software[s] and components, as well as provide a (necessary) level of abstraction of the complexity of the tool (and organize the tool's functionalities [and user development(s) / utilization(s)]). This tool is hoped to be accessible to any person[s] who wish to either begin work in the field of autonomous system development, or those who have the desire to continue existing work, using a platform that is both easy to use, yet sufficiently rigorous to accomplish complex tasks and testing[s].
*Side Note: The original project "AVSIM" had a greater emphasis on the "software" side of simulation, and were there not necessarily plans for hardware integration (via ROS for instance) in the original plans and system schematics. AgentSim aims at, however, device integration inclusion (of sensors devices [LiDAR, camera, etc...], OBUs (vehicle CAN interpreters, V2V communication devices), as well as custom hardware). This device integration functionality is ideally desired to smoothly integrate into the procedural generation pipeline (via the [to be discussed] procedural asset generation pipeline, all of which shall be located in the Device Platform Module Component. The "hardware" side of the project has a bit of its own hierarchy (and again that was not the original focus of any this work [more so was the intent to produce a simulated test bed for device creation, rather than existing device integration (an intent which has now changed)]), and a separate development cycle is in progress [a bit further in the future] for this aspect of the project.
Each facet / aspect [component] of the procedural generation pipeline is divided and implemented by "Plugin" module components. This "plugin" model forms a composable means for which to put pieces of the procedural generation pipeline together. This contributed to one of the unique ideas that the entire platform / ecosystem desires to have, namely, the ability for parts and pieces of the ecosystem to be added or removed, included or not included, and generally for the entire system to be a mutable, adaptability, and flexibility "puzzle board" (in the good way of course - essentially "plug and play" [a bit pleonastic]), where the paradigms of modularity and extensibility are key. The Run-time component is mainly for the "real time"or "online" related functionalities, and the developer component is for "off line" functionality, whose details can be seen in the figure. The editor component is dependent upon the visualization platform used (for example whether Unity, Unreal 4 or Blender is being used).
The management system of the GE component handles many aspects of the simulation platform (as can be seen in the diagram below). The managers coordinates between all of the various elements in the platform (more details shall be available in the executive summary of AgentSim to come...).
The basic file structure of the core system of the GE component is depicted below. One of the reasons for introducing these diagrams is to give the reader a brief glimpse into the code structure (as the code structure (of the entire AgentSim Ecosystem [and AVSIM]) found in the Github repositories is (mostly) directly reflected by these diagrams).
The basic GE component data and workflow model was originally meant for the development of control systems for control vehicles (hence the emphasis on optimal control (and reinforcement learning) methods in the pipeline.As per the discussion of [2], did a larger purpose (use case) become clear for the AgentSim Ecosystem (a topic digression of which shall not be discussed here). In general, is the data and workflow presented in the diagrams below a reflection of the original (AVSIM) system purpose mainly
These data and workflow diagrams are by no means a final product (pretty much the same for the entire project so far), but are just an initial sketch of what how data is to move through the system ideally. TODO: fix data flow discrepancies
The final depicted diagram is a quick side note about the idea of context in reference to the scenario-agent-environment relation of the procedural generation pipeline. Procedurally generated environments are aimed at providing the means to create complex scenarios (for experimentation, etc...) and so, in general, is there a desire to form contexts (a generalization of [complex] scenarios). Contexts are abstractly built and represented by levels, which are further decomposed into scenes, worlds, actors (static and dynamic elements), relevant scenario state (information) and other (user defined) externalities. Experiments, specific testing (the general formation of [a series of] tests), applications, services, and things of this nature are all captured and encapsulated within context (components).
*Note: Scenarios are likely to follow OpenScenario specification - http://www.openscenario.org/project.html
The GE components (which implements the core of the procedural generation pipeline) is intended to act a base of development, and is not only to be used as the central 3D rendering and physics / graphics ... . The figure below gives just a brief idea of how the simulation component brings together several other components to be used within a given 3D scene (or scenario) visualization.

*Side Note: on the Development Cycle

The process of development within the AgentSim Ecosystem is currently divided into three phases of abstraction. The focus of this work was on the development cycle contained within the simulation component of the ecosystem, but in future work (upon reaching maturation) shall the AgentSim ecosystem be intended to host three concurrent cycles of development (testing and design) all of which shall contribute to the larger, "master" cycle.

*Side Note: on the Device Integration

Some video references to further demonstrate the ideas of this side note:
https://www.youtube.com/watch?v=zpzIGC7qmjI
https://www.youtube.com/watch?v=92i7wh6qdjQ
Naturally when working with simulation software that have the purpose of simulating actual physical systems, does one seek to find means of connecting the simulation with the "real world" / physical model[s]. At the current moment, is a goal to include a method of device (sensors: LIDAR, Radar, IMU, [*OBU communications units are another topic...], etc...) communications and integration via the ROS (robot operating system) "platform". The idea (pretty much as one might expect, and is also being implemented by others [in other various contexts]) is to use ROS (and perhaps the python API - this idea has yet to reach full stability, as it is the most novel concept of this work [as software was more of the principle focus rather than hardware]), as a means for allowing real-world devices and simulated devices to communicate and (homomorphically) reflect one another (basically, what a machine does in the real-world is reflected / displayed [commensurately] in a simulated environment). Ideally, is there the hope that one can create custom [real-world] hardware and test the efficacy inside of a simulated environment (for instance, one could "plug" the real-world device into a simulated machine [autonomous vehicle for example]). Also, is there a goal to allow for the facilitation of transfer learning between real-world and simulated machines (either of sensors or of controller [or any hardware device really]), where training (or testing_ in either type of environment can be transferred into the alternate environment (and vice versa). All of this functionality is to encapsulated by the Device Platform software (and hardware) component module. As has been stated, the ecosystem hopes to not only provide a simulation platform, but also a platform for )(hardware) device creation (such as the creation of physical complex autonomous system and corresponding controller units), and does the ecosystem hope to provide a networking / communication infrastructure to allow for all of these different development cycles to commune with one another (in the formation of the entire AgentSim development cycle [see the above side note], all via the AgentSim AS Infrastructure component [see AgentSim Ecosystem diagram]).
*Eventually to extend to development of Bio-metric Sensor Systems / Networks (BSS / BSN) of / for mobility agents: https://github.com/dominic-eaton10/BreathApp (also see [45])

Conclusion and Future Work

As was stated in the introduction, this is preliminary work and there is still many things to be done toward completing the final pipeline / algorithm of procedural generation of complex and diverse environments. Upon the finishing the creation of environments of static and dynamic elements, can work begin (or continue as there is concurrently work being done on developing complex autonomous systems) on the development of the complex autonomous mobility systems themselves. Here considerations such as networking (vehicle ad-hoc networks and V2V communications), distributed vehicular systems, vehicular edge computing, distributed machine learning applications (for optimal control purposes mainly), and many other services and applications shall be integrated into the environment of automation, whose foundation is based on the procedural generation pipeline of this work. Also, there is concurrent work being done on the more theoretic and philosophic nature of complex autonomous systems, which is summarized in [1]. The reference [1] gives an overview of the entire project, and it is hope that the work done here with procedural generation shall be of significant utility in achieving the final, overall objectives outlined in [1]. A key point of the procedural generation (for which shall be the focus of future work) shall be that related to the procedural generation of dynamic agent details (so procedural animation, materials, and other [detailed] assets, with emphasis on dynamic procedural agents [this is the "Procedural Character Generation" App module component in the Github "PG" repository]).
Eventually the platform shall include also functionality for Federated Learning / Multi-Task Learning applications (for autonomous vehicles) and applications for vehicular data privacy (for distributed machine learning applications mostly [using the concepts of differential privacy, multi-party computations, moment of accountant, and the "Privacy API" (see [27-32])]).
*vehicular social community (VSCs) application development (in the DES component) is also of interest, as well as incentive mechanism design protocols (not just for VSC, but generally for resource management among distributed agents [another notion to come from the "Blockchain" technology (mentioned next) ]).
Though security and privacy (models, techniques, algorithms) have not been mentioned explicitly in this work, these subject areas are nonetheless of paramount consideration in the final design of the Ecosystem. In conjunction with these, are concepts of distributed systems (and "Blockchain" / state machine replication concepts) also likely to be integrated into the final product. So, even further down the road, does this work seek to find integration (and reconciliation) with concepts of "Blockchain" Technology (here, reference is made to not just the data-structure, but also the protocols and intuitive notions that this technology provides [e.g. "Democratization", "Decentralized Coordination / Management", smart / incomplete contracts, decentralized autonomous organizations, automated economics and governance ["for" machines and not strictly / necessarily "of" machines], etc...]), (see [33-35]), and it is hoped that the concepts of Blockchain, Distributed Machine Learning (along with Privacy and Security notions), Multi-Agent Behavioral Control, Procedural Generation, (and several other topics as well [all of which is to be discussed in significantly greater detail in future works]), can all be smoothly integrated into a cohesive (and ideally elegant / concise) final product of a complete and comprehensive complex autonomous agent (for providing ground, road-based, commercial, mobility services).
Upon reaching greater maturity, it is hope that the work done on this project shall allow for a more standardized (and significantly easier) process for creating complete and comprehensive systems of autonomous complex agents (or network of agents), with the emphasis on the design of optimal control mechanisms for such systems (for the purposes of performing and providing various complex services and flexible applications [for humanity the enhancement of human mobility here, but more broadly,for the general improvement of all human life and its many "components". ]). Much is still to be done, but it is hoped that this work is a solid step in a "good" / "forward" direction.
*A Conclusory note: A point is to made clear that the final AgentSim product hopes to be truly platform/software agnostic. AgentSim seeks to act as a substrate that binds together the many tools, methodologies, and processes involved with the creation of complex autonomous systems (whether these tools involve the use of Unity, Unreal 4, Blender, tensorflow, openAI Gym, Omnet++, etc...).
Next Articles in the series (to come):
Toward Autonomous Agent Simulation: https://connect.unity.com/p/toward-autonomous-agent-simulation
Toward the Simulation of Vehicular Flocks (Crowds): https://medium.com/@domnic_eaton/toward-the-simulation-of-vehicular-flocks-crowds-d5e339a46fab
Toward Decentralized Complex Autonomous Organizations: https://medium.com/the-av-net/toward-decentralized-complex-autonomous-organizations-11f414313341
(addendum) The Complex Autonomous City: https://medium.com/the-av-net/the-complex-autonomous-city-25a6bcce815a
The "self-driving car" and the "decentralized complex autonomous organization" both seem to represent applications of what is thought to be a new class of technology and theory surrounding the complex autonomous system. Both are sub component of an even larger, enveloping system, that shares similar characteristics even if at differing scales. The [possible high degree of ] relatedness and inter-connectedness between these two systems (later to include "The Complex Autonomous City") seems to make them ideal candidates for study and development, at least in this work, and it is thought that a tool of sufficient complexity (for now AgentSim, but in the future is it believed that a large host and diversity of complex tools shall come into existence to accomplish such a task [the tool-set for development of these complex systems shall itself be a complex system as well...] ) shall enable the comprehensive study and development of such applications and systems.

References

  1. https://medium.com/the-av-net/the-autonomous-agent-a-story-part-i-3017a81d46ed
  2. https://medium.com/the-av-net/correspondences-on-agents-1-d2738d4bf765
  3. Smelik, Ruben. (2019). A Declarative Approach to Procedural Generation of Virtual Worlds.
  4. Elinder, Tobias General Methods for the Generation of Seamless Procedural Cities (2017) In LU-CS-EX 2017-30 EDA920 20171 [a principle procedural generation reference...]
  5. Kelly, George & Mccabe, Hugh. (2006). A survey of procedural techniques for city generation. Institute of Technology Blanchardstown Journal. 14.
  6. Yoav I. H. Parish and Pascal Müller. 2001. Procedural modeling of cities. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH '01). ACM, New York, NY, USA, 301-308. DOI: https://doi.org/10.1145/383259.383292 [a principle procedural generation reference...]
  7. KELLY, G. AND MCCABE, H. 2007. Citygen: An interactive system for procedural city generation. In Fifth International Conference on Game Design and Technology. 8–16.
  8. Hendrikx, Mark & Meijer, Sebastiaan & Van Der Velden, Joeri & Iosup, Alexandru. (2013). Procedural Content Generation for Games: A Survey. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMCCAP). 9. 10.1145/2422956.2422957.
  9. Ruben M. Smelik, Tim Tutenel, Rafael Bidarra, and Bedrich Benes. 2014. A Survey on Procedural Modelling for Virtual Worlds. Comput. Graph. Forum 33, 6 (September 2014), 31-50. DOI: https://doi.org/10.1111/cgf.12276
  10. Najahi, Yakin, "Urbis Terram - Designing and Implementing a Procedural City Generation Tool for Unity3D Game Engine" (2017). Masters Theses (All Theses, All Years). 398. https://digitalcommons.wpi.edu/etd-theses/398
  11. Hwang, Rachel. “Procedural Building Generation with Grammars.” YouTube, 9 Nov. 2017, youtu.be/t-VUpX-xVo4.
  12. Dang, Minh. “Interactive Design of Probability Density Functions for Shape Grammars.” YouTube, YouTube, 23 Sept. 2015, www.youtube.com/watch?v=Flz-cIadPP0.
  13. Minh Dang, Stefan Lienhard, Duygu Ceylan, Boris Neubert, Peter Wonka, and Mark Pauly. 2015. Interactive design of probability density functions for shape grammars. ACM Trans. Graph. 34, 6, Article 206 (October 2015), 13 pages. DOI: https://doi.org/10.1145/2816795.2818069
  14. Schwarz, M., & Wonka, P. (2015). Practical grammar-based procedural modeling of architecture. In SIGGRAPH Asia 2015 Courses, SA 2015 [13] Association for Computing Machinery, Inc. https://doi.org/10.1145/2818143.2818152
  15. Guoning Chen, Gregory Esch, Peter Wonka, Pascal Müller, and Eugene Zhang. 2008. Interactive procedural street modeling. In ACM SIGGRAPH 2008 papers (SIGGRAPH '08). ACM, New York, NY, USA, Article 103, 10 pages. DOI: https://doi.org/10.1145/1399504.1360702 [A primary reference point for tensor based modelling of procedural roads (in conjunction with a series of papers / theses regarding tensor field visualization...)]
  16. Grøn Øyvind, Hervik Sigbjørn; Einstein's General Theory of Relativity; [TODO: Complete reference] [mathematical "descriptionary" machinery reference... ]
  17. Ares Lagae, Sylvain Lefebvre, Rob Cook, Tony Derose, George Drettakis, et al.. A Survey of Procedural Noise Functions. Computer Graphics Forum, Wiley, 2010, 29 (8), pp.2579-2600. <10.1111/j.1467-8659.2010.01827.x>. <hal-00920177> [the only work that I have found so far that gives any form of formalized, general description of coherent (procedural) nose...]
  18. T.F. Fwa, The Handbook of Highway Engineering; September 28, 2005 ; 888 Pages - 373 B/W Illustrations ; ISBN 9780849319860 - CAT# 1986
  19. Steven Rabin. 2013. Game AI Pro: Collected Wisdom of Game AI Professionals. A. K. Peters, Ltd., Natick, MA, USA. [Much structure and ideas for the procedural AI component is coming from here...]
  20. Wu, San-Yuan & Sahni, Sartaj. (1994). Fast Algorithms to Partition Simple Rectilinear Polygons. VLSI Design. 1. 10.1155/1994/16075.
  21. Mark de Berg, Otfried Cheong, Marc van Kreveld, and Mark Overmars. 2008. Computational Geometry: Algorithms and Applications (3rd ed. ed.). TELOS, Santa Clara, CA, USA.
  22. Eric Lengyel. 2011. Mathematics for 3D Game Programming and Computer Graphics, Third Edition (3rd ed.). Course Technology Press, Boston, MA, United States.
  23. Robbin, J. W., Salamon, D. A., Introduction to Differential Geometry, ETH, Lecture Notes, preliminary version, January 2011, http://www.math.ethz.ch/~salamon/PREPRINTS/diffgeo2011.pdf.
  24. Bredon, Glen E.; Topology and Geometry (Graduate Texts in Mathematics) Springer New York; 10.1007/978-1-4757-6848-0; ISBN 9781475768480
  25. P. Cvitanovi, R. Artuso, R. Mainieri, G. Tanner and G. Vattay, { Chaos: Classical and Quantum}, {ChaosBook.org} (Niels Bohr Institute, Copenhagen 2016)
  26. Yoav Shoham and Kevin Leyton-Brown. 2008. Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, New York, NY, USA. [principle textbook reference on "current-ish"views of multi agent system theories]
  27. Dwork, Cynthia and Vitaly Feldman. “Privacy-preserving Prediction.” COLT (2018). [machine learning + privacy + security series reference...]
  28. McMahan, H. Brendan, et al. "Learning differentially private recurrent language models." arXiv preprint arXiv:1710.06963 (2017). [machine learning + privacy + security series reference...]
  29. Bonawitz, Keith, et al. "Practical secure aggregation for privacy-preserving machine learning." Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2017. [machine learning + privacy + security series reference...]
  30. Abadi, Martin, et al. "Deep learning with differential privacy." Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2016. [machine learning + privacy + security series reference...]
  31. Bassily, Raef, Om Thakkar, and Abhradeep Thakurta. "Model-agnostic private learning via stability." arXiv preprint arXiv:1803.05101 (2018). [machine learning + privacy + security series reference...]
  32. Weng, J., et al. "Deepchain: Auditable and privacy-preserving deep learning with blockchain-based incentive." Cryptology ePrint Archive, Report 2018/679 (2018). [machine learning + privacy + security series reference...]
  33. Garay, Juan, Aggelos Kiayias, and Nikos Leonardos. "The bitcoin backbone protocol: Analysis and applications." Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer, Berlin, Heidelberg, 2015. [formalization modes reference for "Toward DCAOs..."]
  34. Pass, Rafael, Lior Seeman, and Abhi Shelat. "Analysis of the blockchain protocol in asynchronous networks." Annual International Conference on the Theory and Applications of Cryptographic Techniques. Springer, Cham, 2017. [formalization modes reference for "Toward DCAOs..."]
  35. Pass, Rafael, and Elaine Shi. "The sleepy model of consensus." International Conference on the Theory and Application of Cryptology and Information Security. Springer, Cham, 2017. [formalization modes reference for "Toward DCAOs..."]
  36. Bagdasaryan, Eugene, et al. "How to backdoor federated learning." arXiv preprint arXiv:1807.00459 (2018).
  37. Yao, Andrew Chi-Chih. "Protocols for secure computations." FOCS. Vol. 82. 1982
  38. Soni, Aakash and Huosheng Hu. “Formation Control for a Fleet of Autonomous Ground Vehicles: A Survey.” Robotics 7 (2018): 67. [formalization modes reference for "Toward Vehicular Crowds..."]
  39. Akbari Mistani, Pouria & Pakravan, Samira & Gibou, Frederic. (2018). Towards a tensor network representation of complex systems. [formalization modes reference for "Toward Vehicular Crowds..."]
  40. Yousefpour, Ashkan et al. “All One Needs to Know about Fog Computing and Related Edge Computing Paradigms: A Complete Survey.” CoRR abs/1808.05283 (2018): n. pag.
  41. Pathak, Deepak, et al. "Curiosity-driven exploration by self-supervised prediction." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2017.
  42. Juliani, Arthur & Berges, Vincent-Pierre & Vckay, Esh & Gao, Yuan & Henry, Hunter & Mattar, Marwan & Lange, Danny. (2018). Unity: A General Platform for Intelligent Agents.
  43. Dosovitskiy, Alexey & Ros, German & Codevilla, Felipe & Lopez, Antonio & Koltun, Vladlen. (2017). CARLA: An Open Urban Driving Simulator. a
  44. Kenney, John. (2011). Dedicated Short-Range Communications (DSRC) Standards in the United States. Proceedings of the IEEE. 99. 1162 - 1182. 10.1109/JPROC.2011.2132790.
  45. Ramadan Gad, Nawal El-Fishawy, AYMAN EL-SAYED and M. Zorkany, “Multi-Biometric Systems: A State of the Art Survey and Research Directions” International Journal of Advanced Computer Science and Applications(IJACSA), 6(6), 2015. http://dx.doi.org/10.14569/IJACSA.2015.060618
  46. J. B. Kenney, "Dedicated Short-Range Communications (DSRC) Standards in the United States," in Proceedings of the IEEE, vol. 99, no. 7, pp. 1162-1182, July 2011. doi: 10.1109/JPROC.2011.2132790
  47. https://medium.com/the-av-net/correspondences-on-consistency-3eaf8215e842
  48. Stuart Russell and Peter Norvig. 2009. Artificial Intelligence: A Modern Approach (3rd ed.). Prentice Hall Press, Upper Saddle River, NJ, USA.

Further Reading and Resources

Further Readings

  1. Hawking, S., & Ellis, G. (1973). The Large Scale Structure of Space-Time (Cambridge Monographs on Mathematical Physics). Cambridge: Cambridge University Press. doi:10.1017/CBO9780511524646
  2. J. W. Gibbs; "On the equilibrium of heterogeneous substances"; Am J Sci December 1878 Series 3 Vol. 16:441-458; doi:10.2475/ajs.s3-16.96.441
  3. N. Bourbaki, Elements of Mathematics Algebra I: Chapters 1-3 [TODO: Complete reference]
  4. Sheldon M. Ross. 2006. Introduction to Probability Models, Ninth Edition. Academic Press, Inc., Orlando, FL, USA.
  5. Kardar, M. (2007). Statistical Physics of Particles. Cambridge: Cambridge University Press. doi:10.1017/CBO9780511815898
  6. Dwork, Cynthia, and Aaron Roth. "The algorithmic foundations of differential privacy." Foundations and Trends® in Theoretical Computer Science 9.3–4 (2014): 211-407. [chapter 10, connection to Game Theory...]
  7. Kevin M. Lynch and Frank C. Park. 2017. Modern Robotics: Mechanics, Planning, and Control (1st ed.). Cambridge University Press, New York, NY, USA.
  8. Rajamani, Rajesh. Vehicle dynamics and control. Springer Science & Business Media, 2011.
  9. Nise, Norman S. Control Systems Engineering. Hoboken, NJ: Wiley, 2004. Print.
  10. Alan V. Oppenheim and Ronald W. Schafer. 2009. Discrete-Time Signal Processing (3rd ed.). Prentice Hall Press, Upper Saddle River, NJ, USA.
  11. Alan V. Oppenheim and Ronald W. Schafer. 1989. Discrete-Time Signal Processing. Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
  12. Bernard Sklar. 1988. Digital Communications: Fundamentals and Applications. Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
  13. Charles L Phillips, John Parr, and Eve Riskin. 2007. Signals, Systems, and Transforms (4th ed.). Prentice Hall Press, Upper Saddle River, NJ, USA.
  14. Shah, Devavrat. "Gossip algorithms." Foundations and Trends® in Networking 3.1 (2009): 1-125. [the "contagion" principle of complex adaptive systems]
  15. Ramakumar, Ramachandra. "Engineering Reliability Fundamentals and Applications". Prentice Hall, 2002. [long term reference]

Resources and References

  1. https://www.redblobgames.com/maps/terrain-from-noise/
  2. http://www.terraincomposer.com/
  3. http://www.terraincomposer.com/procedural-terrain/
  4. http://www.terraincomposer.com/worldcomposer/
  5. http://martindevans.me/game-development/2015/12/11/Procedural-Generation-For-Dummies/
  6. http://lup.lub.lu.se/luur/download?func=downloadFile&recordOId=8929185&fileOId=8929189
  7. https://leafletjs.com/
  8. http://www.citygen.net/
  9. https://flow-project.github.io/
  10. https://ray.readthedocs.io/en/latest/rllib.html
  11. https://veins.car2x.org/tutorial/
  12. https://veins.car2x.org/
  13. http://carla.org/
  14. https://ray.readthedocs.io/en/latest/tune.html [automated parameter tuning (ray, rllib)]
  15. http://www.opendrive.org/ [specification]
  16. http://www.openscenario.org/project.html [specification]
  17. https://www.openstreetmap.org/#map=4/38.01/-95.84 [specification]
  18. https://www.openssl.org/docs/ [crypto]
  19. https://aldrin.co/crypto-primitives.html [crypto]
  20. https://www.cryptopp.com/ [crypto]
  21. https://en.wikipedia.org/wiki/Cryptographic_primitive [crypto]
  22. https://developer.android.com/things
  23. https://stackoverflow.com/questions/32491646/how-to-plot-the-heat-map-for-a-given-function-in-python
  24. http://firsttimeprogrammer.blogspot.com/2015/07/the-heat-equation-python-implementation.html

Code Repositories

  1. https://bitbucket.org/martindevans/base-citygeneration/src/87878c33627f?at=default [static element procedural generation - Refactoring]
  2. https://github.com/magnificus/Procedural-Cities [static element procedural generation - Refactoring]
  3. https://github.com/ayron/delaunay
  4. https://github.com/jmespadero/pyDelaunay2D/blob/master/delaunay2D.py
  5. https://github.com/arbonagw/HeliumRain [Menu system (singleton) implementation comes from here - Refactoring]
  6. https://github.com/sheljohn/ndArray [general data structure for nD procedural noise and tensors - Refactoring]
  7. https://github.com/noncomputable/AgentMaps [an interesting dynamic agent generation java-script code base]
  8. https://www.compadre.org/stp/filingcabinet/share.cfm?UID=10986&FID=22841&code=8E844C06A4 [statistical mechanics percolation theory java visualization code base]
  9. https://github.com/mittalgovind/Polygon-Partition [an O(n) polygon partitioning algorithm]
  10. https://github.com/michele-segata/plexe-veins [Refactoring vehicle platoon app for vehicle flocking]
  11. https://github.com/michele-segata/plexe-sumo [ACC vehicle model implementation from Ramanji Book - Refactoring]
  12. https://github.com/flow-project/flow [vehicle controls models - Refactoring]
  13. https://github.com/ray-project/ray
  14. https://github.com/rll/rllab
  15. https://github.com/sommer/veins [discrete even simulation for vehicles (V2V communications, in Omnet++)]
  16. https://github.com/Auburns/FastNoise
  17. https://github.com/ivanfratric/polypartition [polygon partitioning method of Elinder (see [4])]
  18. https://github.com/ue4plugins/StreetMap
  19. https://github.com/NVIDIAGameWorks/PhysX [vehicle physics models - Refactoring]
  20. https://github.com/karim-emara/PREXT [security and privacy models (networking / Omnet++ / DES component) - Refactoring]
  21. https://github.com/openssl/openssl [crypto]
  22. https://github.com/pbugnion/gmaps
  23. https://github.com/ray-project/ray/tree/master/python/ray/tune [automated parameter tuning (ray, rllib)]

Simulation

  1. https://github.com/carla-simulator *[likely to be Refactored...]
  2. https://github.com/Microsoft/AirSim *[likely to be Refactored...]
  3. https://github.com/ApolloAuto/apollo *[likely to be Refactored...]
  4. https://github.com/CagataySonmez/EdgeCloudSim [porting to Omnet++ - Refactoring]
  5. https://github.com/Cloudslab/cloudsim [porting to Omnet++ - Refactoring]
  6. https://github.com/eclipse/sumo [vehicle traffic models - Refactoring]
  7. https://github.com/ray-project/ray/tree/master/python/ray/rllib
  8. https://github.com/ray-project/ray [parallel computing platform...]
  9. https://github.com/rll/rllab
  10. https://gym.openai.com/ [environment intended to be defined as according to this standard (along with OpenDrive / OpenScenario)]
  11. https://github.com/Unity-Technologies/ml-agents/tree/master/docs [a framework that is ideally to be encapsulated into the AI Platform software module component (along with OpenAI Gym standards and Rllib)]
  12. https://github.com/Microsoft/AutonomousDrivingCookbook
  13. https://github.com/tjgerot/AIM4-Simulator
  14. Gu, Tianyu. “A Lightweight Simulator for Autonomous Driving Motion Planning Development.” ICIS 2015 (2015). [Carnegie Melon Simulator]
  15. Li, Wei et al. “AADS: Augmented Autonomous Driving Simulation using Data-driven Algorithms.” CoRR abs/1901.07849 (2019): n. pag. [University of Maryland Simulator]
  16. https://www.faac.com/realtime-technologies/products/ [entire class / suite of simulator[s] / simulation tools; FAAC Simulatino Platform]
  17. https://www.carsim.com/
  18. https://www.quantumsignal.com/
  19. https://www.nvidia.com/en-us/self-driving-cars/drive-constellation/
  20. http://drivesim.osu.edu/about/ [Ohio State University Simulator]
  21. https://drivingsim.phhp.ufl.edu/ [University of Florida Simulator]
  22. http://vision-traffic.ptvgroup.com/en-us/products/ptv-vissim/
  23. https://www.nsnam.org/ [NS3 network simulation (also vehicular traffic generation)]
  24. https://veins.car2x.org/ [main implementation of vehicular traffic + networking simulation to come from here - Refactoring]
  25. http://www.cs.utexas.edu/~aim/ [University of Texas vehicular intersection management simulator]
  26. https://www.3ds.com/products-services/simulia/products/powerflow/#_ga=2.108989506.1580953142.1555536423-877cc1a0-6157-11e9-b684-3b9eae273682 [Dassault Systèmes Simulation Platform]
  27. https://www.3ds.com/products-services/simulia/products/simpack/#_ga=2.108989506.1580953142.1555536423-877cc1a0-6157-11e9-b684-3b9eae273682 [Dassault Systèmes Simulation Platform]
articles:
  1. https://www.businessinsider.com/waymo-engineer-explains-why-testing-self-driving-cars-virtually-is-critical-2018-8
  2. https://blogs.unity3d.com/tag/simviz/
  3. https://blogs.unity3d.com/2018/11/08/jump-start-your-autonomous-simulation-development-with-unitys-simviz-solution-template/
  4. https://www.slideshare.net/unity3d/autonomous-vehicle-development-with-unity

Standards, Specifications and Regulations

  1. https://www.sae.org/standards/development/dsrc SAE J2735
  2. SAE J2945.1
  3. IEEE WAVE 802.11p
  4. IEEE 1609(.2 -.4) [security]
  5. DSRC protocol[s] (see [46]) [overall encompassing technology (standard)]

Project Code Repositories and more Information

More information about this work (and the large encompassing work) can be found at the medium publication sites:
https://medium.com/the-av-net
https://medium.com/the-av-net/the-autonomous-agent-a-story-part-i-3017a81d46ed
https://medium.com/the-av-net/correspondences-on-agents-1-d2738d4bf765
Code for the generated figures as well as code for the entire procedural generation project can be found at the Github repositories (all initial testing work is done in python / Blender (for "quick and dirty" development and testing), and then converted [*still currently being converted] into c++ / java (java is for visualization testing purposes mainly)):
https://github.com/dominic-eaton10/WorldProcGen-TestCode [To be converted into c++ and integrated into MathLib - Refactoring]
https://github.com/dominic-eaton10/PG
https://github.com/dominic-eaton10/MathLib
https://github.com/dominic-eaton10?tab=repositories [contains the code for all the entire AgentSim Project (though many repositories have yet to be filled as of late ["complete the project" is on the TODO list...])]
The Original Procedural Generation project code (The "GE" module component, procedural world generation, and procedural AI were all done in Unreal 4 initially (this was the old project called "AVSIM", and the new project is now called "AgentSim"), and this work is to be eventually be converted to Unity [and Blender Game Engine (BGE) at some future point in time ...]) all of which can be found at the following Github repositories:
https://github.com/dominic-eaton10/AVSIM_Core-AVSIM_Module
https://github.com/dominic-eaton10/AVSIM_Math-AVSIM_Module
https://github.com/dominic-eaton10/BuildingCreator-AVSIM_Module
https://github.com/dominic-eaton10/EigenPlugin-AVSIM_Module
https://github.com/dominic-eaton10/PedestrianCreator-AVSIM_Module
https://github.com/dominic-eaton10/TrafficCreator-AVSIM_Module
https://github.com/dominic-eaton10/TransitCreator-AVSIM_Module
https://github.com/dominic-eaton10/VehicleCreator-AVSIM_Module
*The other AVSIM modules have been omitted due to significant incompleteness.. The PedestrianAsset module (which contains created pedestrian content using Adobe Fuse + Mixamo pipeline) was far to large to be included into a Github repository, and other efforts shall be made to make this content available publicly .

End Notes

*This work is but a single perspective in a much larger work concerning complex autonomous systems (all perspectives are still in a state of "nascency" and the connections between them are still not quite yet clear). Preliminary thoughts and ideas looking from the other perspectives are given below:
  1. Consistency and Internal Simulation: https://medium.com/the-av-net/correspondences-on-consistency-3eaf8215e842
  2. Entropy and Energy (a thermodynamic / bio-chemical connection): https://medium.com/the-av-net/correspondences-on-agents-1-d2738d4bf765
  3. The nature and effects of (psychological / system) [deliberate] abnormalities (a faculty, an origination of [the functionalities of] security, privacy and trust ?): https://medium.com/the-av-net/on-deviance-d31f91100afe
  4. The faculty of Reflection (an origination of the functionality of internal simulation ?): https://medium.com/the-av-net/on-reflection-90903db2047b
  5. The functionalities of security, privacy and trust: https://medium.com/the-av-net/on-trust-e914aa5bb8e7
  6. On The vicissitudes / capriciousness of (the entity of) symbolic representation (on the causes of rises and falls of language): https://medium.com/the-av-net/toward-the-creation-and-destruction-of-language-part-i-1b1c30e3a237
  7. On the nature of (general and self) governance: https://medium.com/the-av-net/on-governance-d6f4889f94dd
  8. The functionality of mobility (of complex autonomous agents): https://medium.com/the-av-net/a-question-of-mobility-4c03d6c79d21
  9. On creating a principle (or at least more complete) narrative for complex autonomous systems (of mobility): https://medium.com/the-av-net/the-autonomous-agent-a-story-part-i-3017a81d46ed
  10. The beginnings of the formation of connections between: Governance, Economics, Finance and Law as well as Complex Autonomous Systems and "Stand Alone Complexes" ("leaderless entities" / starling flock murmurations / stock market entities formation connection ?): https://medium.com/the-av-net/on-decentralization-a-first-thought-3285c5c1a637
  11. On the nature of perception (a look at perception and sensory sensors of modern autonomous vehicles and the connection between the philosophy and biology concepts of "senses" and the further interrelations between complex autonomous systems) https://medium.com/the-av-net/on-perception-9d1d461ffdc4
*Currently, for organizational purposes initially are the multi-dimensionalities of this and related works are exposed by the addition of "the perspectives" sections found generally in preliminary notes sections of all works (though generally the inter-relatedness of these areas and knowledge of other fields is not currently known, and likely these sections are expected to grow and change considerably in times to come.)
*WARNING: Some of the code repositories are still not well organized yet (at all) as the project still in that "raw development" stage, something for which shall be on the top of list of things to do as the project continues further.
*Formal computational model of procedural generation
*FINAL NOTE: As always, this work, its ideas and concepts, are very much open and are continuously changing, growing and developing. Comments, inquiries, questions, concerns, general thoughts, responses, reactions or feedback of any kind are always very much appreciated and welcome.
"It is difficult to make predictions, especially about the future". It is difficult to say what one shall find at the end of all this, but does my own internal simulation model seem to proliferate many possibilities of great auspiciousness for applications that may use such technology that is being described here. I can not say whether or not any of this shall be of any use, or if it shall be (at the very least) a driver for continued work in this field, and can provide significant contributions accordingly. Do I feel it remiss for me not to attempt and follow this path and discover its ends, and I all can do at the present moment is believe that one day, this work shall be some utility to someone. If the fates have begun to spin the thread regarding the destiny of the complex autonomous agent, then I shall, at least for this current moment, believe that the work described here (and all that is to follow) shall be of at least some consideration.
*The Central Work: https://medium.com/the-av-net/the-complex-autonomous-system-4500f10d4a74

A
AgentSim
2
Comments