The technology
Real-time computer graphics
Real-time computer graphics is the subfield of computer graphics focused on producing and analyzing images in real time. The term is most often used in reference to interactive 3D computer graphics, typically using a GPU, with video games the most noticeable users. The term can also refer to anything from rendering an application’s GUI to real-time image processing and image/data analysis.
Although computers have been known from the beginning to be capable of generating 2D images in real-time involving simple lines, images and polygons - 3D computer graphics has always been a daunting task for traditional graphics systems to keep up with the speed necessary for generating fast, good quality 3D computer images onto a display screen.
Principles of real-time 3D computer graphics
The goal of computer graphics is to generate a computer generated image using certain desired metrics. This image is often called a frame. How fast these images or frames are generated in a given second determines the method’s real-timeliness.
One interesting aspect of real-time computer graphics is the way in which it differs from traditional off-line rendering systems (and hence, these are the non-real-time graphics systems); non-real-time graphics typically rely on ray-tracing where the expensive operation of tracing rays from the camera to the world is allowed and can take as much as hours or even days for a single frame. On the other hand, in the case of real-time graphics, the system has less than 1/30th of a second per image. In order to do that, the current systems cannot afford shooting millions or even billions of rays; instead, they rely on the technique of z-buffer triangle rasterization. In this technique, every object is decomposed into individual primitives - the most popular and common one is the triangle (or Polygon). These triangles are then ‘drawn’ or rendered onto the screen one by one. Each of these triangles get positioned, rotated and scaled on the screen and a special hardware (or in the case of an emulator, the software rasterizer) called rasterizer generates the pixels inside each of these triangles. These triangles are then decomposed into further smaller atomic units called pixels (or in computer graphics terminology, aptly called fragments) that are suitable for displaying on a display screen. The pixels are then drawn on the screen using a certain color; current systems are capable of deciding the color that results in these triangles - for e.g. a texture can be used to ‘paint’ onto a triangle, which is simply deciding what color to output at each pixel based on a stored picture; or in a more complex case, at each pixel, one can compute if a certain light is being seen or not resulting in very good shadows (using a technique called shadow mapping).
Thus, real-time graphics is oriented toward providing as much performance as possible for the lowest quality possible for a given class of hardware. Most video-games and simulators fall in this category of real-time graphics. As mentioned above, real-time graphics is currently possible due to the significant recent advancements in these special hardware components called graphics processing units (GPUs). These GPUs are capable of handling millions of triangles per frame and within each such triangle capable of handling millions or even billions of pixels (i.e. generating these pixel colors).
Why real-time computer graphics?
Another interesting difference between real-time and non-real-time graphics is the interactivity desired in real-time graphics. Feedback is typically the main motivation for pushing real-time graphics to its furore. In cases like films, the director has the complete control and determinism of what has to be drawn on each frame, typically involving weeks or even years of decision-making involving a number of people.
In the case of real-time interactive computer graphics, usually a user is in control of what is about to be drawn on the display screen; the user typically uses an input device to provide feedback to the system - for example, wanting to move a character on the screen - and the system decides the next frame based on this particular instance of action. Usually the display is far slower (in terms of the number of frames per second) in responsiveness than the input device (in terms of the input device’s response time measured in ms). In a way this is justified due to the immense difference between the infinitesimal response time generated by a human-being’s motion and the very slow perspective speed of the human-visual system; this results in significant advancements in computer graphics, whereas the advancements in input devices typically take a much longer time to achieve the same state of fundamental advancement (for e.g. the current Wii controller), as these input devices have to be extremely fast in order to be usable.
Another important factor controlling real-time computer graphics is the combination of physics and animation. These techniques largely dictate what is to be drawn on the screen - or more precisely, where to draw certain objects (deciding their position) on the screen. These techniques imitate the behavior (the temporal dimension, not the spatial dimensions) seen in real-world to a degree that is far more realistic than and compensating computer-graphics’ degree of realism.
Visualisation
Visualisation is any technique for creating images, diagrams, or animations to communicate a message. visualisation through visual imagery has been an effective way to communicate both abstract and concrete ideas since the dawn of man.
Interactive visualisation is a branch of graphic visualisation in computer science that studies how humans interact with computers to create graphic illustrations of information and how this process can be made more efficient. In order for visualisation to be considered interactive, it must satisfy two criteria:
Human Input: control of some aspect of the visual representation of information, or of the information being represented, must be available to a human, and
Response Time: changes made by the human must be incorporated into the visualisation in a timely manner. In general, interactive visualisation is considered a soft real-time task.
One particular type of interactive visualisation is virtual reality (VR), where the visual representation of information is presented using an immersive display device such as a stereo projector. VR is also characterized by the use of a spatial metaphor, where some aspect of the information is represented in three dimensions so that humans can explore the information as if it were present (where instead it was remote), sized appropriately (where instead it was on a much smaller or larger scale than humans can sense directly), or had shape (where instead it might be completely abstract).
Another type of interactive visualisation is collaborative visualisation, in which multiple people interact with the same computer visualisation to communicate their ideas to each other or to explore information cooperatively. Frequently, collaborative visualisation is used when people are physically separated. Using several networked computers, the same visualisation can be presented to each person simultaneously.
Simulation
A simulation is an imitation of some real thing, state of affairs, or process. The act of simulating something generally entails representing certain key characteristics or behaviours of a selected physical or abstract system.
Simulation is used in many contexts, including the modeling of natural systems or human systems in order to gain insight into their functioning. Other contexts include simulation of technology for performance optimization, safety engineering, testing, training and education. Simulation can be used to show the eventual real effects of alternative conditions and courses of action.
Key issues in simulation include acquisition of valid source information about the referent, selection of key characteristics and behaviours, the use of simplifying approximations and assumptions within the simulation, and fidelity and validity of the simulation outcomes.
- Physical simulation refers to simulation in which physical objects are substituted for the real thing. These physical objects are often chosen because they are smaller or cheaper than the actual object or system.
- Interactive simulation is a special kind of physical simulation, often referred to as a human in the loop simulation, in which physical simulations include human operators, such as in a flight simulator or a driving simulator. Human in the loop simulations can include a computer simulation, e.g. when simulating a nuclear-power-plant crisis, to verify personnel can react to the simulated computer displays.
A computer simulation (or "sim") is an attempt to model a real-life or hypothetical situation on a computer so that it can be studied to see how the system works. By changing variables, predictions may be made about the behavior of the system. Computer simulation has become a useful part of modeling many natural systems in physics, chemistry and biology, and human systems in economics and social science as well as in engineering to gain insight into the operation of those systems.
Traditionally, the formal modeling of systems has been via a mathematical model, which attempts to find analytical solutions enabling the prediction of the behavior of the system from a set of parameters and initial conditions. Computer simulation is often used as an adjunct to, or substitution for, modeling systems for which simple closed form analytic solutions are not possible. There are many different types of computer simulation, the common feature they all share is the attempt to generate a sample of representative scenarios for a model in which a complete enumeration of all possible states would be prohibitive or impossible.
Simulation is often used in the training of civilian and military personnel. This usually occurs when it is prohibitively expensive or simply too dangerous to allow trainees to use the real equipment in the real world. In such situations they will spend time learning valuable lessons in a "safe" virtual environment. Often the convenience is to permit mistakes during training for a safety-critical system.
Training simulations typically come in one of three categories:
- "live" simulation (where real people use simulated (or "dummy") equipment in the real world);
- "virtual" simulation (where real people use simulated equipment in a simulated world (or "virtual environment")), or
- "constructive" simulation (where simulated people use simulated equipment in a simulated environment). Constructive simulation is often referred to as "war gaming" since it bears some resemblance to table-top war games in which players command armies of soldiers and equipment that move around a board.