GPipe/Tutorial

From HaskellWiki

< GPipe(Difference between revisions)

Current revision

1 Introduction
2 Creating a window
3 PrimitiveStreams
4 FragmentStreams
5 FrameBuffers
6 Screenshot

1 Introduction

In GPipe, you'll primary work with these four types of data on the GPU:

Let's walk our way through an simple example as I explain how you work with these types. This example requires GPipe version 1.4.1 or later. This page is formatted as a literate Haskell page, simply save it as "box.lhs" and then type

ghc --make –O box.lhs
box

at the prompt to see a spinning box. You’ll also need an image named "myPicture.jpg" in the same directory (I used a picture of some wooden planks).

> module Main where
 
> import Graphics.GPipe
> import Graphics.GPipe.Texture.Load
> import qualified Data.Vec as Vec
> import Data.Vec.Nat
> import Data.Monoid
> import Data.IORef
> import Graphics.UI.GLUT
>     (Window,
>     mainLoop,
>     postRedisplay,
>     idleCallback,
>     getArgsAndInitialize,
>     ($=))

Besides GPipe, this example also uses the GPipe-TextureLoad package for loading textures from disc. GLUT is used in GPipe for window management and the main loop.

2 Creating a window

We start by defining the

main

function.

> main :: IO ()
> main = do
>     getArgsAndInitialize
>     tex <- loadTexture RGB8 "myPicture.jpg"
>     angleRef <- newIORef 0.0
>     newWindow "Spinning box" (100:.100:.()) (800:.600:.()) (renderFrame tex angleRef) initWindow
>     mainLoop       
 
> renderFrame :: Texture2D RGBFormat -> IORef Float -> Vec2 Int -> IO (FrameBuffer RGBFormat () ())
> renderFrame tex angleRef size = do
>     angle <- readIORef angleRef
>     writeIORef angleRef ((angle + 0.005) `mod'` (2*pi))
>     return $ cubeFrameBuffer tex angle size
 
> initWindow :: Window -> IO ()
> initWindow win = idleCallback $= Just (postRedisplay (Just win))

First we set up GLUT, and load a texture from disc via the GPipe-TextureLoad package function

loadTexture

. In this example we're going to animate a spinning box, and for that we put an angle in an

IORef

so that we can update it between frames. We then create a window with newWindow. When the window is created,

initWindow

registers this window as being continously redisplayed in the idle loop. At each frame, the

IO

action

renderFrame tex angleRef size

is run. In this function the angle is incremented with 0.005 (reseted each lap), and a

FrameBuffer

is created and returned to be displayed in the window. But before I explain

FrameBuffer

s, let's jump to the start of the graphics pipeline instead.

3 PrimitiveStreams

The graphics pipeline starts with creating primitives such as triangles on the GPU.Let's create a box with six sides, each made up of two triangles each.

> cube :: PrimitiveStream Triangle (Vec3 (Vertex Float), Vec3 (Vertex Float), Vec2 (Vertex Float))
> cube = mconcat [sidePosX, sideNegX, sidePosY, sideNegY, sidePosZ, sideNegZ]
 
> sidePosX = toGPUStream TriangleStrip $ zip3 [1:.0:.0:.(), 1:.1:.0:.(), 1:.0:.1:.(), 1:.1:.1:.()] (repeat (1:.0:.0:.()))    uvCoords
> sideNegX = toGPUStream TriangleStrip $ zip3 [0:.0:.1:.(), 0:.1:.1:.(), 0:.0:.0:.(), 0:.1:.0:.()] (repeat ((-1):.0:.0:.())) uvCoords
> sidePosY = toGPUStream TriangleStrip $ zip3 [0:.1:.1:.(), 1:.1:.1:.(), 0:.1:.0:.(), 1:.1:.0:.()] (repeat (0:.1:.0:.()))    uvCoords
> sideNegY = toGPUStream TriangleStrip $ zip3 [0:.0:.0:.(), 1:.0:.0:.(), 0:.0:.1:.(), 1:.0:.1:.()] (repeat (0:.(-1):.0:.())) uvCoords
> sidePosZ = toGPUStream TriangleStrip $ zip3 [1:.0:.1:.(), 1:.1:.1:.(), 0:.0:.1:.(), 0:.1:.1:.()] (repeat (0:.0:.1:.()))    uvCoords
> sideNegZ = toGPUStream TriangleStrip $ zip3 [0:.0:.0:.(), 0:.1:.0:.(), 1:.0:.0:.(), 1:.1:.0:.()] (repeat (0:.0:.(-1):.())) uvCoords
 
> uvCoords = [0:.0:.(), 0:.1:.(), 1:.0:.(), 1:.1:.()]

Every side of the box is created from a regular list of four elements each, where each element is a tuple with three vectors: a position, a normal and an uv-coordinate. These lists of vertices are then turned into PrimitiveStreams on the GPU by toGPUStream that in our case creates triangle strips from the vertices, i.e 2 triangles from 4 vertices. Refer to the OpenGl specification on how triangle strips and the other topologies works.

All six sides are then concatenated together into a cube. We can see that the type of the cube is a

PrimitiveStream

of Triangles where each vertex is a tuple of three vectors, just as the lists we started with. One big difference is that those vectors now are made up of

Vertex Float

s instead of

Float

s since they are now on the GPU.

The cube is defined in model-space, i.e where positions and normals are relative the cube. We now want to rotate that cube using a variable angle and project the whole thing with a perspective projection, as it is seen through a camera 2 units down the z-axis.

> transformedCube :: Float -> Vec2 Int -> PrimitiveStream Triangle (Vec4 (Vertex Float), (Vec3 (Vertex Float), Vec2 (Vertex Float)))
> transformedCube angle size = fmap (transform angle size) cube
 
> transform angle (width:.height:.()) (pos, norm, uv) = (transformedPos, (transformedNorm, uv))
>     where
>         modelMat = rotationVec (normalize (1:.0.5:.0.3:.())) angle `multmm` translation (-0.5)
>         viewMat = translation (-(0:.0:.2:.())) 
>         projMat = perspective 1 100 (pi/3) (fromIntegral width / fromIntegral height)
>         viewProjMat = projMat `multmm` viewMat
>         transformedPos = toGPU (viewProjMat `multmm` modelMat) `multmv` (homPoint pos :: Vec4 (Vertex Float))
>         transformedNorm = toGPU (Vec.map (Vec.take n3) $ Vec.take n3 modelMat) `multmv` norm

When applying a function on the

PrimitiveStream

using

fmap

, that function will be executed on the GPU using vertex shaders. The toGPU function transforms normal values like

Float

s into GPU-values like

Vertex Float

so it can be used with the vertices of the

PrimitiveStream

4 FragmentStreams

To render the primitives on the screen, we must first turn them into pixel fragments. This is called rasterization and in our example done by the function rasterizeFront, which transforms

PrimitiveStream

s into FragmentStreams.

> rasterizedCube :: Float -> Vec2 Int -> FragmentStream (Vec3 (Fragment Float), Vec2 (Fragment Float))
> rasterizedCube angle size = rasterizeFront $ transformedCube angle size

In the rasterization process, values of type

Vertex Float

are turned into values of type

Fragment Float

For each fragment, we now want to give it a color from the texture we initially loaded, as well as light it with a directional light coming from the camera.

> litCube :: Texture2D RGBFormat -> Float -> Vec2 Int -> FragmentStream (Color RGBFormat (Fragment Float))
> litCube tex angle size = fmap (enlight tex) $ rasterizedCube angle size
 
> enlight tex (norm, uv) = RGB (c * Vec.vec (norm `dot` toGPU (0:.0:.1:.())))
>     where RGB c = sample (Sampler Linear Wrap) tex uv

Using

fmap

on a

FragmentStream

will execute a function on the GPU using fragment shaders. The function sample is used for sampling the texture we have loaded, using the fragment's interpolated uv-coordinates and a sampler state. Once we have a

FragmentStream

Color

s, we can paint those fragments onto a

FrameBuffer

5 FrameBuffers

A FrameBuffer is a 2D image in which fragments from

FragmentStream

s are painted. A

FrameBuffer

may contain any combination of a color buffer, a depth buffer and a stencil buffer. Besides being shown in windows,

FrameBuffer

s may also be saved to memory or converted to textures, thus enabling multi pass rendering. A

FrameBuffer

has no defined size, but take the size of the window when shown, or are given a size when saved to memory or converted to a texture. And so finally, we paint the fragments we have created onto a black

FrameBuffer

. By this we use paintColor without any blending or color masking.

> cubeFrameBuffer :: Texture2D RGBFormat -> Float -> Vec2 Int -> FrameBuffer RGBFormat () ()
> cubeFrameBuffer tex angle size = paintSolid (litCube tex angle size) emptyFrameBuffer
 
> paintSolid = paintColor NoBlending (RGB $ Vec.vec True)
> emptyFrameBuffer = newFrameBufferColor (RGB 0)

This

FrameBuffer

is the one we return from the

renderFrame

action we defined at the top.

6 Screenshot

Retrieved from "http://www.haskell.org/haskellwiki/GPipe/Tutorial"

Category: Tutorials

@@ Line 2: / Line 2: @@
 In GPipe, you'll primary work with these four types of data on the GPU:
-* [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Primitive.html#t:PrimitiveStream <hask>PrimitiveStream</hask>]
+* [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Primitive.html#t:PrimitiveStream <code>PrimitiveStream</code>]
-* [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Fragment.html#t:FragmentStream <hask>FragmentStream</hask>]
+* [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Fragment.html#t:FragmentStream <code>FragmentStream</code>]
-* [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-FrameBuffer.html#t:FrameBuffer <hask>FrameBuffer</hask>]
+* [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-FrameBuffer.html#t:FrameBuffer <code>FrameBuffer</code>]
-* [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Texture.html <hask>Texture</hask>]
+* [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Texture.html <code>Texture</code>]
-Let's walk our way through an simple example as I explain how you work with these types. This example requires GPipe version 1.2.1 or later.
+Let's walk our way through an simple example as I explain how you work with these types. This example requires GPipe version 1.4.1 or later.
 This page is formatted as a literate Haskell page, simply save it as &quot;<tt>box.lhs</tt>&quot; and then type
 <pre>
@@ Line 23: / Line 23: @@
 > import qualified Data.Vec as Vec
 > import Data.Vec.Nat
-> import Data.Vec.LinAlg.Transform3D
 > import Data.Monoid
 > import Data.IORef
@@ Line 36: / Line 35: @@
 </haskell>
-Besides [http://hackage.haskell.org/package/GPipe GPipe], this example also uses the [http://hackage.haskell.org/package/Vec-Transform Vec-Transform package] for the transformation matrices, and the [http://hackage.haskell.org/package/GPipe-TextureLoad GPipe-TextureLoad package] for loading textures from disc. [http://hackage.haskell.org/package/GLUT GLUT] is used in GPipe for window management and the main loop.
+Besides [http://hackage.haskell.org/package/GPipe GPipe], this example also uses the [http://hackage.haskell.org/package/GPipe-TextureLoad GPipe-TextureLoad package] for loading textures from disc. [http://hackage.haskell.org/package/GLUT GLUT] is used in GPipe for window management and the main loop.
 == Creating a window ==
@@ Line 64: / Line 62: @@
 </haskell>
-First we set up GLUT, and load a texture from disc via the [http://hackage.haskell.org/package/GPipe-TextureLoad GPipe-TextureLoad package] function <hask>loadTexture</hask>. In this example we're going to animate a spinning box, and for that we put an angle in an <hask>IORef</hask> so that we can update it between frames. We then create a window with [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-FrameBuffer.html#v:newWindow <hask>newWindow</hask>]. When the window is created, <hask>initWindow</hask> registers this window as being continously redisplayed in the idle loop. At each frame, the <hask>IO</hask> action <hask>renderFrame tex angleRef size</hask> is run. In this function the angle is incremented with 0.005 (reseted each lap), and a <hask>FrameBuffer</hask> is created and returned to be displayed in the window. But before I explain <hask>FrameBuffer</hask>s, let's jump to the start of the graphics pipeline instead.
+First we set up GLUT, and load a texture from disc via the [http://hackage.haskell.org/package/GPipe-TextureLoad GPipe-TextureLoad package] function <hask>loadTexture</hask>. In this example we're going to animate a spinning box, and for that we put an angle in an <hask>IORef</hask> so that we can update it between frames. We then create a window with [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-FrameBuffer.html#v:newWindow <code>newWindow</code>]. When the window is created, <hask>initWindow</hask> registers this window as being continously redisplayed in the idle loop. At each frame, the <hask>IO</hask> action <hask>renderFrame tex angleRef size</hask> is run. In this function the angle is incremented with 0.005 (reseted each lap), and a <hask>FrameBuffer</hask> is created and returned to be displayed in the window. But before I explain <hask>FrameBuffer</hask>s, let's jump to the start of the graphics pipeline instead.
@@ Line 87: / Line 85: @@
 </haskell>
-Every side of the box is created from a regular list of four elements each, where each element is a tuple with three vectors: a position, a normal and an uv-coordinate. These lists of vertices are then turned into [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Primitive.html#t:PrimitiveStream <hask>PrimitiveStream</hask>]s on the GPU by [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Primitive.html#v:toGPUStream <hask>toGPUStream</hask>] that in our case creates triangle strips from the vertices, i.e 2 triangles from 4 vertices. Refer to the OpenGl specification on how triangle strips and the other topologies works.
+Every side of the box is created from a regular list of four elements each, where each element is a tuple with three vectors: a position, a normal and an uv-coordinate. These lists of vertices are then turned into [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Primitive.html#t:PrimitiveStream <code>PrimitiveStream</code>]s on the GPU by [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Primitive.html#v:toGPUStream <code>toGPUStream</code>] that in our case creates triangle strips from the vertices, i.e 2 triangles from 4 vertices. Refer to the OpenGl specification on how triangle strips and the other topologies works.
-All six sides are then concatenated together into a cube. We can see that the type of the cube is a <hask>PrimitiveStream</hask> of [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Primitive.html#t:Triangle <hask>Triangle</hask>]s where each vertex is a tuple of three vectors, just as the lists we started with. One big difference is that those vectors now are made up of <hask>Vertex Float</hask>s instead of <hask>Float</hask>s since they are now on the GPU.
+All six sides are then concatenated together into a cube. We can see that the type of the cube is a <hask>PrimitiveStream</hask> of [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Primitive.html#t:Triangle <code>Triangle</code>]s where each vertex is a tuple of three vectors, just as the lists we started with. One big difference is that those vectors now are made up of <hask>Vertex Float</hask>s instead of <hask>Float</hask>s since they are now on the GPU.
 The cube is defined in model-space, i.e where positions and normals are relative the cube. We now want to rotate that cube using a variable angle and project the whole thing with a perspective projection, as it is seen through a camera 2 units down the z-axis.
@@ Line 104: / Line 102: @@
 >         projMat = perspective 1 100 (pi/3) (fromIntegral width / fromIntegral height)
 >         viewProjMat = projMat `multmm` viewMat
->         transformedPos = toGPU (viewProjMat `multmm` modelMat) `multmv` homPoint pos
+>         transformedPos = toGPU (viewProjMat `multmm` modelMat) `multmv` (homPoint pos :: Vec4 (Vertex Float))
->         transformedNorm = toGPU (Vec.map (Vec.take n3) $ Vec.take n3 $ modelMat) `multmv` norm
+>         transformedNorm = toGPU (Vec.map (Vec.take n3) $ Vec.take n3 modelMat) `multmv` norm
 </haskell>
-When applying a function on the <hask>PrimitiveStream</hask> using <hask>fmap</hask>, that function will be executed on the GPU using vertex shaders. The [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream.html#v:toGPU <hask>toGPU</hask>] function transforms normal values like <hask>Float</hask>s into GPU-values like <hask>Vertex Float</hask> so it can be used with the vertices of the <hask>PrimitiveStream</hask>.
+When applying a function on the <hask>PrimitiveStream</hask> using <hask>fmap</hask>, that function will be executed on the GPU using vertex shaders. The [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream.html#v:toGPU <code>toGPU</code>] function transforms normal values like <hask>Float</hask>s into GPU-values like <hask>Vertex Float</hask> so it can be used with the vertices of the <hask>PrimitiveStream</hask>.
 == FragmentStreams ==
-To render the primitives on the screen, we must first turn them into pixel fragments. This called rasterization and in our example done by the function [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Fragment.html#v:rasterizeFront <hask>rasterizeFront</hask>], which transforms <hask>PrimitiveStream</hask>s into [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Fragment.html#t:FragmentStream <hask>FragmentStream</hask>]s.
+To render the primitives on the screen, we must first turn them into pixel fragments. This is called rasterization and in our example done by the function [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Fragment.html#v:rasterizeFront <code>rasterizeFront</code>], which transforms <hask>PrimitiveStream</hask>s into [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-Stream-Fragment.html#t:FragmentStream <code>FragmentStream</code>]s.
 <haskell>
@@ Line 137: / Line 135: @@
 </haskell>
-Using <hask>fmap</hask> on a <hask>FragmentStream</hask> will execute a function on the GPU using fragment shaders. The function [http://hackage.haskell.org/packages/archive/GPipe/1.1.3/doc/html/Graphics-GPipe-Texture.html#v:sample <hask>sample</hask>] is used for sampling the texture we have loaded, using the fragment's interpolated uv-coordinates and a sampler state.
+Using <hask>fmap</hask> on a <hask>FragmentStream</hask> will execute a function on the GPU using fragment shaders. The function [http://hackage.haskell.org/packages/archive/GPipe/1.1.3/doc/html/Graphics-GPipe-Texture.html#v:sample <code>sample</code>] is used for sampling the texture we have loaded, using the fragment's interpolated uv-coordinates and a sampler state.
 Once we have a <hask>FragmentStream</hask> of <hask>Color</hask>s, we can paint those fragments onto a <hask>FrameBuffer</hask>.
 == FrameBuffers ==
-A [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-FrameBuffer.html#t:FrameBuffer <hask>FrameBuffer</hask>] is a 2D image in which fragments from <hask>FragmentStream</hask>s are painted. A <hask>FrameBuffer</hask> may contain any combination of a color buffer, a depth buffer and a stencil buffer. Besides being shown in windows, <hask>FrameBuffer</hask>s may also be saved to memory or converted to textures, thus enabling multi pass rendering. A <hask>FrameBuffer</hask> has no defined size, but take the size of the window when shown, or are given a size when saved to memory or converted to a texture.
+A [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-FrameBuffer.html#t:FrameBuffer <code>FrameBuffer</code>] is a 2D image in which fragments from <hask>FragmentStream</hask>s are painted. A <hask>FrameBuffer</hask> may contain any combination of a color buffer, a depth buffer and a stencil buffer. Besides being shown in windows, <hask>FrameBuffer</hask>s may also be saved to memory or converted to textures, thus enabling multi pass rendering. A <hask>FrameBuffer</hask> has no defined size, but take the size of the window when shown, or are given a size when saved to memory or converted to a texture.
-And so finally, we paint the fragments we have created onto a black <hask>FrameBuffer</hask>. By this we use [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-FrameBuffer.html#v:paintColor <hask>paintColor</hask>] without any blending or color masking.
+And so finally, we paint the fragments we have created onto a black <hask>FrameBuffer</hask>. By this we use [http://hackage.haskell.org/packages/archive/GPipe/latest/doc/html/Graphics-GPipe-FrameBuffer.html#v:paintColor <code>paintColor</code>] without any blending or color masking.
 <haskell>

Personal tools

Views