Yhc web service

Part of Yhc (Download)

Yhc Web Service

Purpose

Yhc Web Service is a web-based tool to use Yhc's Javascript back end without installation of any additional software on user's computer. Only a standards-compilant web browser is needed. The service allows to submit some Haskell source code for compilation into Javascript, and generates a xhtml1 page which is temporarily stored. A link is provided to load the generated page into user's web browser. In general, Yhc web service's user interface is similar to one of a pastebin.

Hosting

The Service is currently available at the following URL: http://www.golubovsky.org:5984/static/yhcws/MainGUI.html

Technical overview

Yhc Web Service is built around CouchDB which itself is written in Erlang. CouchDB uses JSON as encoding for queries and responses. A special daemon regularly polls the database for updates. If a new Haskell source is submitted for compilation as a new document, the daemon extracts it from the database, and invokes Yhc to transform it along with modules imported into linked Yhc Core. Next, Javascript Backend is invoked, to convert the linked Core into Javascript. Finally, using the XHTML template (user-supplied or standard), a Web page is built. Output from the compiler and other programs called during compilation is also collected. Results of compilation are stored in the database as attachments to the document which contains the submitted Haskell source.

The service runs a slightly modified version of SearchPath to chase imports across the Internet. This removes the limitation on the number of modules compiled simultaneously, while only one Haskell module may be submitted via user interface.

The Service runs under Linux inside a 32-bit virtual machine which is an instance of KVM administered by KVMADM on a 64-bit Linux host. Since CouchDB currently lacks any access control to its databases, Squid is used as reverse proxy with proper access control settings.

Users guide

User interface

Initially, the Service displays the following form in the web browser window. Each row of the form corresponds to a single pastebin entry. The links column contains icons that correspond to possible user actions that can be performed over each document. The author column identifies the submitter (author name is arbitrary: the Service does not authrnticate users). The time column shows the submission time (UTC). The title column shows the title that author gave to the entry. The status column shows whether the code submitted with the entry is being comipled, or that compilation was successful, or it failed.

The view Haskell source icon is always present regardless of current status of an entry. Clicking on it shows the source code submitted with the entry. Clicking on the Close button restores the document browser.

The view compilation log and view error log icons when clicked, show either the compilation log, or error log depending on success or failure of compilation. These icons appear only after compilation is complete (successful or failed). Similarly, the Close button restores the document browser.

Clicking on the edit source code icon opens the new entry form with the source of the corresponding entry. Thus, source code may be edited and resubmitted. The service does not keep track of source code revisions: each time source is edited, it is submitted as a new entry. This icon is always present with each entry.

The view compiled page icon opens a new browser window where the compiled web page is loaded. This icon appears only after successful compilation.

On the document browser form, buttons << Prev and Next >> "turn pages" of the pastebin. Upon startup, the document browser shows the most recent entries. The New Entry button invokes the new entry form with all fields empty, so new code may be entered from scratch.

On the new entry form, the Clear All button clears all entry fields, Browse switches back to the document browser (showing the most recent entries) regardless of whether the new code was submitted or not. The Submit button sends the user input for compilation. After pressing the Submit button, the new entry form remains active, and submission results appear in the Source field. If submission was successful (that is, successfully stored in the database), new document's DocId appears in the Source field. Otherwise (e. g. HTTP error) error message appears in the Source field. Compilation results never appear in the new entry form.

Yhc Web Service does not limit the size of Haskell code entered.

Typical usage scenario

1. In the document browser, press the New Entry button. The new entry form appears.
2. Type or paste into the Source field whatever is desired to be submitted. Also fill in the Author and Title form. If empty, these fields will be initialized with default values at the moment of submission.
3. Press the Submit button. Make sure that new DocId appears in the Source field.
4. Press the Browse button to switch back to the document browser. Locate the entry just submitted by author and title. Note the compilation status.
5. Press the << Prev button several times with approximately 5 second interval and wait until compilation status changes to 'Success' or 'Error'.
6. If the status becomes 'Error', observe the error log by clicking on icon, then click on icon to correct the source code in error.
7. If the status becomes 'Success', click on icon to load the compiled page.

Build it yourself

There is no unified way to build the instance of the Service as it depends largely upon other software distributions. The version presently online was built based on Arch Linux Duke, however Arch Linux users are advised to see this message.

In order to build the Service, make sure CouchDB is built and running; follow instructions.

Check out the Yhc darcs repo:

darcs get http://darcs.haskell.org/yhc

Change to the root of Yhc source tree. Build Yhc:

scons core=1 build && scons prefix=/where/you/install install

Stay at the root of Yhc source tree. Build the Javascript backend:

(cd src/translator/js; make all cdbtools install)

At this point CouchDB must be running. It is assumed that everything runs at the same host, that is, the database is accessed at http://localhost:5984.

Stay at the root of Yhc source tree. Build the Service:

(cd src/yhcws; make all init-db install)

At the moment of making init-db, you will be asked to remove and re-create several databases in the running CouchDB instance. Make sure you are not going to lose any valuable data.

Everything will be installed relatively to Yhc binary location. That is, if yhc is located as /usr/bin/yhc, all binaries associated with the Service will be installed in the same directory. Make sure that this directory is on your path.

Create a user account under which the compiler will run. All that such a user needs is just a home directory.

Start the service by running ywsd. This program acts as a compile job scheduler polling the database for updates every second, and running Yhc if there is anything new to compile.

As the database is accessed every second, logs of the database itself and also http access logs grow fast. It makes sense to cut those logs frequently as they are of little value: use an utility provided with your distro, or write your own, similar to logcut shell script found at src/yhcws relatively to the root of Yhc source tree.

It however saves you a lot of time just to use the online version of the Service.

Programming considerations

Documentation

DOM and Haskell Web Toolkit API documentation generated with Haddock from the most current sources is available at http://www.golubovsky.org:5984/_utils/yhcws/index.html. Please note that not everything described in that documentation is available with Javascript backend. In particular, IO monad is not available: interaction with user is done by means of DOM function calls in continuation passing style (CPS).

Compilation of multiple modules

Only a single module program may be pasted in the New Entry Form. Such limitation, if not worked around, would seriously affect usability of the Service.

The Service uses a slightly modified SearchPath program to chase module imports across the Internet. Thus, if a multi-module program is needed to be compiled, the same technique as used in HaPPS will be involved.

The original SearchPath uses -i command line options to specify URLs of mapfiles (see SearchPath usage guide). The Yhc Web Service expects these addresses to be specified as language pragmas. The pragma to specify an URL of a mapfile is named SP_MAP, and an absolute URL of a mapfile should follow before the pragma comment closing.

For example, suppose a module TestMod is to be imported. Module location is http://www.golubovsky.org/repos/ywtest/TestMod.hs, and its contents follows:

module TestMod where

msg = "You can see this: spyhc worked."

A mapfile to address the module import should be placed as http://www.golubovsky.org/repos/ywtest/test.map, and its contents follows:

TestMod http://www.golubovsky.org/repos/ywtest

In order to import this module it is necessary to include the comment shown below anywhere in the pasted code:

{-# SP_MAP http://www.golubovsky.org/repos/ywtest/test.map #-}

The Service will recognize the pragma in the pasted code, and will invoke the modified SearchPath (spyhc) with proper command-line parameters.

Number of SP_MAP pragmas in the pasted code is not limited. One appearance of the pragma defines one mapfile URL.

Below is the pasteable code example which imports another module using the SP_MAP pragma:

-- begin pasteable code --
{-# SP_MAP http://www.golubovsky.org/repos/ywtest/test.map #-}

module HelloWorldHsWTK where

import DOM.Level2.HTMLDivElement
import Graphics.UI.HsWTK
import TestMod

main = docBodyC (mkDiv |<< textP msg)
-- end   pasteable code --

Compilation log of this code follows:

• Compilation of HelloWorldHsWTK was successful
• Compile log follows

curl -s -f -L -x http://localhost:5984 http://localhost:5984/static/yhcws/template.html -o template.html /home2/YHCWS/bin/spyhc -i. -i$(dirname $(dirname `which yhc`))/lib/yws -i$(dirname $(dirname `which yhc`))/lib/haskell -ihttp://www.golubovsky.org/repos/ywtest/test.map yhc --cpp --linkcore --no-bytecode HelloWorldHsWTK.hs Compiling TestMod ( .sp-cache/TestMod.hs ) Compiling HelloWorldHsWTK ( HelloWorldHsWTK.hs ) Loading Core for YHC.Internal Loading Core for YHC.Primitive Loading Core for Data.Ratio Loading Core for Prelude Loading Core for Foreign.C Loading Core for Foreign.C.String Loading Core for Foreign.Storable Loading Core for Foreign.Word Loading Core for Numeric Loading Core for Data._CharNumeric Loading Core for Data.Char Loading Core for Data.Array Loading Core for YHC.Exception Loading Core for Data.Ix Loading Core for Foreign.Ptr Loading Core for Foreign.Int Loading Core for Foreign.Util Loading Core for Foreign.C.Error Loading Core for YHC.ErrNo Loading Core for Foreign.C.Types Loading Core for Control.Monad Loading Core for System.IO Loading Core for System.IO.Unsafe Loading Core for Data.IORef Loading Core for YHC.IORef Loading Core for Foreign Loading Core for Foreign.Marshal Loading Core for Foreign.Marshal.Alloc Loading Core for Foreign.Marshal.Utils Loading Core for Foreign.StablePtr Loading Core for Foreign.ForeignPtr Loading Core for Debug.Trace Loading Core for System.Exit Loading Core for PreludeAux Loading Core for Graphics.UI.HsWTK Loading Core for DOM.Level2.Html2 Loading Core for UnsafeJS Loading Core for CPS Loading Core for DOM.Level2.Dom Loading Core for Data.Maybe Loading Core for DOM.Level2.Node Loading Core for DOM.Level2.Document Loading Core for Control.Concurrent.JSThreads Loading Core for Data.JSRef Loading Core for CDOM.Level2.DomUtils Loading Core for DOM.Level2.Css Loading Core for DOM.Level2.Stylesheets Loading Core for DOM.Level2.Views Loading Core for DOM.Level2.Events Loading Core for DOM.Level2.HTMLDocument Loading Core for CDOM.Level2.Events Loading Core for DOM.Level2.Element Loading Core for DOM.Level2.HTMLElement Loading Core for DOM.Level2.HTMLTextAreaElement Loading Core for DOM.Level2.HTMLImageElement Loading Core for DOM.Level2.HTMLParamElement Loading Core for DOM.Level2.HTMLOptionElement Loading Core for DOM.Level2.HTMLButtonElement Loading Core for DOM.Level2.HTMLSelectElement Loading Core for DOM.Level2.HTMLInputElement Loading Core for DOM.Level2.HTMLDivElement Loading Core for DOM.Level2.KeyEvent Linking... "yhc --includes=. --includes=/YHCWS/lib/yws --includes=/YHCWS/lib/haskell --includes=.sp-cache HelloWorldHsWTK.hs --cpp --linkcore --no-bytecode" $(dirname $(dirname `which yhc`))/bin/ycr2js HelloWorldHsWTK.yca HelloWorldHsWTK\;main > HelloWorldHsWTK.js $(dirname $(dirname `which yhc`))/bin/pgbuild -t template.html -o HelloWorldHsWTK.html -T "HelloWorldHsWTK" -e $(dirname $(dirname `which yhc`))/lib/javascript/Runtime.js -e $(dirname $(dirname `which yhc`))/lib/javascript/JUCD.js -e HelloWorldHsWTK.js \ --onload="exprEval(funIdx['HelloWorldHsWTK\;main'])" rm HelloWorldHsWTK.js HelloWorldHsWTK.yca

Modules mentioned in the online documentation are available for import without the need to use any SP_MAP pragmas.

Custom web page templates

Additionally, by using the HTML_TEMPLATE pragma, custom Web page template may be specified. The pragma should have an URL of a downloadable (X)HTML file after the pragma name. There should be only one occurrence of this pragma in the pasted code. Default Web page template is contained here.

Information about developer

Dmitry Golubovsky developed and currently maintains the Yhc Javascript backend and the Yhc Web Service. Please send e-mail to golubovsky at gmail dot com for any questions, comments, or constructive critics.