Scala class at Stanford

I am pleased to announce that Ben Newman, Julien Wetterwald, and I will be teaching CS94SI: Cross-Paradigm Programming with Scala, a student-initiated course at Stanford University this coming Spring. As the name suggests, the course will focus on how Scala can be used to program with several programming paradigms (functional, object-oriented, concurrent, domain-specific, etc.).

If you have any experience teaching Scala, in an academic setting or otherwise, I'd love to hear about your experiences at jorge.ortiz@gmail.com.

<Lecturer Details>
Ben is pursuing a Masters degree in Computer Science at Stanford with a specialization in Software Theory. He has a passion for programming languages and has extensive teaching experience, having been a teaching assistant for CS106A, CS107, and CS156 at Stanford.

Julien is also pursuing a Masters degree in Computer Science at Stanford with a specialization in Software Theory. His main areas of interest are programming language design and compilers. As an undergraduate at EPFL, he was a teaching assistant for Martin Odersky's class Programmation IV, taught in Scala. He has used Scala to write a compiler and a domain-specific language for graphical user interfaces.

Jorge graduated from Stanford with a Bachelors in Symbolic Systems in June 2007. During his time there he helped teach CS106A, CS106X, and CS107. He is a contributor for http://scala-blogs.org/ and the lift web framework, written in Scala. He uses lift and Scala every day at work.
</Lecturer Details>

<Course Details>
No previous exposure to Scala is required, though we do ask that students have studied at least two programming languages in an academic setting (CS107 is recommended; CS 242 may increase your appreciation for the class). Students will be expected to complete a final project of their choosing. The one-unit class will meet Wednesdays from 4:15-5:45pm in Gates 260. Stanford students can register for CS94SI on Axess.
</Course Details>

lift 0.5, now with Blueprint CSS beauty, jQuery excellence, and JSON goodness

Folks,

I am pleased to announce lift version 0.5.

lift is an expressive and elegant framework for writing web applications. lift stresses the importance of security, maintainability, scalability and performance, while allowing for high levels of developer productivity.

Demo: http://demo.liftweb.net

New in version 0.5:

• jlift JavaScript utilities and associated binder helpers


• Blueprint CSS built in (version 0.6 of Blueprint CSS) so it can be served via /classpath


• XML -> JavaScript bindings so that JSON -> XHTML rendering can be defined using Scala XML literals, but the resulting JavaScript code can be sent to the browser to do mass rendering of JSON objects


• JSON support including: - S.buildJSONFunc - JSONHandler class - Lots of fancy JavaScript expression building


• Parser Combinator Helper routines


• ByList query term


• Support for serving certain common files (e.g., jQuery, Blueprint CSS) right from the JAR/WAR file


• MappedText and MapperFakeClob classes for storing very long text fields in the database

Changes include:

• Revised the example site to use Blueprint CSS for style


• When lift is running in test mode, the default form generators will insert lift:field_name attribute into the form fields to aid in testing


• Added json2.js (and minified version) Enhanced the resource serving stuff so that: - The path is defined in LiftServlet and changable (defaults to "classpath") - Moved the resource location from the root to "toserve" (can be changed) to avoid any possible way of serving up classes - The white list is a Partial Function for more flexibility - The white list by default contains jquery.js and json.js - There's a re-writer to re-write the request into an actual file (so jquery.js gets re-written to jquery-1.2.2-min.js)


• MappedField.toForm now returns a Can[NodeSeq ] so a field can be omitted from a form


• Added LocInfo and LocInfoVal to SiteMap for storing CSS and other tidbits about each menu item


• Fixed the 'Welcome to the your project' type-o


• Further abstracted the whole logging thing. There's a default log4j setup that can be override The LiftLogger trait is made concrete in Log4JLogger, but the generic LiftLogger creation is done by function that can be modified in LogBoot: var loggerByName: String => LiftLogger = _logger var loggerByClass: Class => LiftLogger = _loggerCls One can insert a new logging system into lift by replacing the loggerByName and loggerByClass functions with appropriate functions that return a LiftLogger.


• Extensive updates to the <lift:xxx /> tag mechanism that uses tag labels as snippets and support for adding arbitrary tag handling to lift.

The lift community has grown to over 400 people. There are a number of lift-based project in development including http://much4.us which makes use of lift's Comet, AJAX, and JSON support.

Thanks to the excellent, talented team of lift committers including Jorge Ortiz who was build-master for this release, Steve Jenson who keeps adding practical things to lift, and the rest of the lift team.

Please take a look at lift 0.5, join the lift community, and give us some feedback!

Thanks,

David

From Regexps to Regexp Patterns, still functional

Here now is the long overdue follow-up to "RegExp but no state machine". Remember, we were talking about regular expressions: state machines come to mind, and we hurry to the cabinet with the imperative chainsaw, but it turns out that some scalpel-like pure functions can be used to cut the problem as well.

Binding Variables

Quite often, we are not just interested in the boolean answer to the question "did this input match the regexp?". We also want to know the parts that matched parts of the regexp, as in "if the file ends in .txt, put its name without the extension into variable x". In other words, we want to bind variables to parts of the input, depending on how the input matched. Regexps that contain variables deserve to get a beautiful and original name: let us call them regexp patterns.

You want to see an example that is not buried in natural language? We can do that... I will use Scala syntax, but keep in mind that regexp patterns in Scala are
undergoing a redesign - these example are just here to explain regexp patterns.

 
def just_the_name(name:Seq[Char]) = name match {
  case Seq(filename @ _*, '.', 't', 'x', 't') => filename
}

Matching regexp patterns is a bit more involved than matching regexps. That is not apparent from the above, tame example, but what about something like this

def is_backup(name:Seq[Char]) = name match {
  case Seq(_*, bak @ (() | ('.','b','a','k'))) => !bak.isEmpty
}

There are two ways that the input name can match the regexp patterns, a fact that is cleverly used to return a boolean to the caller. When there are many or-patterns and "wildcard_star" patterns, the number of possibilities can grow very large.

Back to Business

Maybe regexp patterns will be resurrected soon in Scala, for now, let us focus on the problem and its functional solution. We will write a Scala program that matches regular expression patterns.

Here is code to represent regexp patterns. It uses the RegExp classes defined before (a complete listing that you can compile in one go is provided at the end).

abstract class Pat[A]
// v is the variable (represented as Int), w the prefix that was
//  already seen, r the remaining part we yet have to match
case class PVar[A](v:Int, word:RegExp.Word[A], re:RegExp[A]) extends Pat[A] {
  override def toString = "x"+v+"@"+re.toString
}
case class PPair[A](left:Pat[A], right:Pat[A]) extends Pat[A]
case class PChoice[A](left:Pat[A], right:Pat[A]) extends Pat[A]
case class PatNil[A] extends Pat[A]
case class PatCons[B](hd:B, tl:List[B]) extends Pat[B]

As the comment says, these pattern representation is interesting: it does not only represent regexp patterns, but also partial matches of inputs. When we write down a regexp pattern, the "word" field of PVar will be empty. During the matching, as input becomes available, the different possibilities of going through the input will correspond to different instances of Pat.

Let us try how that would look for the is_backup example. To keep things short, we turn the pattern into x@'A', y@(()|'B')). We match the input A against the pattern, and see how we would go about this. We freely add the empty sequence () where it is appropriate.

Is the first letter of A consumed by x@'A', y@(()|'B'))? Yes, and it is bound to x.
Is () consumed by y@(()|'B'))? Yes, and nothing is bound y and we are done.

Now the same with AB

Is the first letter of AB consumed by x@'A', y@(()|'B'))? Yes, and it is bound to x.
Is the first letter of B consumed by y@(()|'B'))? Yes, and B is bound y.
Is () consumed by ()? yes and we are done.

So we want to step character by character through the input, and
keep track of the bindings by rewriting our patterns -- this is were the partial derivatives come into play. A function pdPat will rewrite a pattern with a partial derivative (the partDeriv function was explained before, in "RegExp but no state machine".)

  def pdPat[A](pat:Pat[A],c:A):Pat[A] = 
    pat match {
      case PVar(x, w, r) => PVar(x, w:::List(c), partDeriv(r, c))
      case PPair(p1, p2) => 
        val pn = PPair(pdPat(p1, c), p2)
        if (acceptsEmpty(strip(p1)))
          PChoice(pn, PPair(mkEmpPat(p1), pdPat(p2, c)))
        else
          pn
      case PChoice(p1, p2) => PChoice(pdPat(p1, c), pdPat(p2, c))
    }

In short, this function takes a regexp pattern and a character. It goes through the regexp pattern and constructs the partial derivative of its underlying regexp. When it needs to keep track of a binding (case PVar), it does so by appending the character at the end. It uses the functions mkEmpPat (which means matching against empty input and makes the regexps disappear or fail in a regexp pattern, but keeps the bindings that were computed before) and strip, which just gives us the underlying regexp and ignores all variable binding stuff that might be going on.

Exercise: write a function that computes the result of accepts_empty(strip(p)) directly, without constructing a regexp first.

The Full Monty

Here now, without further ado, the full example (including stuff from the first part). As always, it might be instructive to run to try out with your own examples, execute step-wise in your IDE debugger (Eclipse and Netbeans), or, the classic, insert lots of println statements.

Note that different possibilities of feeding an input character to a choice are tracked by return a list of environments. The "longest match" and the "shortest match" policies can be recovered by selecting a special element of the list of environments that is the end result of running the patMatch function.

object RegExpPat {

  // from part 1

  type Word[A] = List[A]

  def matchRegExp[A](re:RegExp[A], w:Word[A]): Boolean =
    w match {
      case List() => acceptsEmpty(re)
      case c::w1  => matchRegExp(partDeriv(re, c), w1)
    }

  def acceptsEmpty[A](re:RegExp[A]): Boolean =
    re match {
      case Phi() => false
      case Empty() => true
      case Choice(r1,r2) => acceptsEmpty(r1) || acceptsEmpty(r2)
      case Seq(r1,r2) => acceptsEmpty(r1) && acceptsEmpty(r2)
      case Star(r) => true
      case L(_) => false
    }

  def partDeriv[A](re:RegExp[A], c:A): RegExp[A] = {
    (re, c) match {
      case (Phi(), _)                => Phi()
      case (Empty(), _)              => Phi()
      case (L(c1), c2) if (c1 == c2) => Empty()
      case (L(c1), _)                => Phi()
      case (Choice(r1,r2), c)        => Choice(partDeriv(r1,c),partDeriv(r2,c))
      case (Seq(r1,r2), c)           =>
        val rn = Seq(partDeriv(r1,c),r2)
        if (acceptsEmpty(r1)) Choice(rn, partDeriv(r2,c)) else rn
      case (rs @ Star(r), c)         => Seq(partDeriv(r,c),rs)
    }
  }

  abstract class RegExp[A]

  case class Phi[A]() extends RegExp[A]
  case class Empty[A]() extends RegExp[A]
  case class L[A](letter:A) extends RegExp[A]
  case class Choice[A](left:RegExp[A], right:RegExp[A]) extends RegExp[A]
  case class Seq[A](left:RegExp[A], right:RegExp[A]) extends RegExp[A]
  case class Star[A](exp:RegExp[A]) extends RegExp[A]

  // part 2

  abstract class Pat[A]
  // vx is the variable (represented as Int), w the prefix that was
  //  already seen, r the remaining part we yet have to match
  case class PVar[A](v:Int, word:Word[A], re:RegExp[A]) extends Pat[A] {
    override def toString = "x"+v+"@"+re.toString
  }
case class PPair[A](left:Pat[A], right:Pat[A]) extends Pat[A]
  case class PChoice[A](left:Pat[A], right:Pat[A]) extends Pat[A]
  case class PatNil[A] extends Pat[A]
  case class PatCons[B](hd:B, tl:List[B]) extends Pat[B]

  // combinator
  def pat_var[A](n: Int, r:RegExp[A]) = PVar(n,List(),r)

  // binding
  type Env[a] = List[(Int,Word[a])]

  def patMatch[A](pat:Pat[A], word:Word[A]):List[Env[A]] = {
    (pat,word) match {
      case (p, c::w) =>
        patMatch(pdPat(pat, c), w)
      case (PVar(x,w,r),List()) =>
        if (acceptsEmpty(r)) List(List((x,w))) else List()
      case (PChoice(p1,p2), List()) =>
        patMatch(p1, List()):::patMatch(p2, List())
      case (PPair(p1,p2), List()) =>
        for(xs <- patMatch(p1, List()); ys <- patMatch(p2, List())) yield xs:::ys
    }
  }

  def longPatMatch[A](pat:Pat[A], word:Word[A]):Option[Env[A]] =
    patMatch(pat,word) match {
      case env::_ => Some(env)
      case _      => None
    }

  def shortPatMatch[A](pat:Pat[A], word:Word[A]):Option[Env[A]] =
    patMatch(pat,word) match {
      case List() => None
      case xs     => Some(xs.last)
    }

  def pdPat[A](pat:Pat[A],c:A):Pat[A] =
    pat match {
      case PVar(x, w, r) => PVar(x, w:::List(c), partDeriv(r, c))
      case PPair(p1, p2) =>
        val pn = PPair(pdPat(p1, c), p2)
        if (acceptsEmpty(strip(p1)))
          PChoice(pn, PPair(mkEmpPat(p1), pdPat(p2, c)))
        else
          pn
      case PChoice(p1, p2) => PChoice(pdPat(p1, c), pdPat(p2, c))
    }

  def strip[A](pat:Pat[A]):RegExp[A] =
    pat match {
      case PVar(_,w,r) => r
      case PPair(p1,p2) => Seq(strip(p1),strip(p2))
      case PChoice(p1,p2) => Choice (strip(p1),strip(p2))
    }

  def mkEmpPat[A](pat:Pat[A]):Pat[A] =
    pat match {
      case PVar(x,w,r) =>
        if (acceptsEmpty(r))
          PVar(x, w, Empty[A]())
        else
          PVar(x, w, Phi[A]())
      case PPair(p1,p2) => PPair(mkEmpPat(p1), mkEmpPat(p2))
      case PChoice(p1,p2) => PChoice (mkEmpPat(p1), mkEmpPat(p2))
    }

  // example for testing
  val the_choice = Choice(Empty(), L('B'))

  val ex = PPair(PVar(1, List(), Star(L('A'))), PVar(2, List(), the_choice))

  def main(args:Array[String]) {
   println(longPatMatch(ex, List('A')))
   println(longPatMatch(ex, List('A','B')))
  }
}

Oh, did I mention that there is no mutable state involved?

Credit: The functional implementation described here follows Kenny Zhuo Ming Lu and Martin Sulzmann. A toast to them! Several people, including myself, found similar solutions, but they are the heroic ones that implemented and proved it all.

Guide to the Scala Community

One of the most intriguing things about Scala is the people who are working with it. I have been following Scala mailing lists over a year, and was impressed by the helpfulness and positive attitude of the people involved. Very few are the times I've seen personal attacks, if any. Also the people at EPFL who are responsible for creation of Scala have given lots of insight why things are done way they are, or why is feature X not implemented. They have even changed Scala rapidly to support other people's ideas.

This blog entry is mainly meant for a Scala beginner as a guide. I try to familiarize you with different ways of interaction with other's in Scala related things, and to get you know about user written faciliation tools and (upcoming) libraries. Any suggestions how to extend this post are appreciated, of course.

First of all, there are the mailing lists. At the moment there are many of them, and it's hard to tell sometimes to which should one post, because the ambiguity of division. But better to post somewhere than to nowhere. Here are the most important:

The main list is scala, and it's for general Scala related discussion, e.g. standard library, language changes, or implementation issues.

scala-user is mostly for newbie questions, and also for specific technology related discussion, also stories of experience of Scala programming.

scala-debate is mostly for discussion that is specific in kind, in which not many people are interested. It's also excellent place to discuss programming theory, future and ideas of improvement for Scala. The discussion seems to be there the most relaxed of all lists, and lots of 'lambda the ultimate' kind of persons are lurking there.

To see guidelines how to interact with the above mentioned and other mailing lists see this page.

The mailing lists are the best places for you to ask questions, or give suggestions, and of course help others. But if you have found a bug e.g. in the compiler, the lists are rarely the best place to announce them. You should just see if there exists a bug report in Trac, and if not, create one. It won't go unnoticed, and will most definitely be appreciated! But in special circumstances it's ok to discuss them too, or ask whether you have found a bug or not.

Please don't get offended if nobody answers your question; it's likely that the question is hard to answer, or it was sent at an unfortunate time, and was forgotten. It's not crime to ask again...

For those who like to chat, the irc channel #scala @ freenode is the place to go. There's at least couple of active guys who try to give their best in helping you. Again, please be patient with your question.

The blogs about Scala are of course not to be forgotten. Here's the most interesting ones I've come up with:

• Tony Morris, a functional programming enthusiastic, has written interesting Scala software and given nice introductions to subjects like implicits: http://blog.tmorris.net/.
• David Pollak talks about Scala webframework lift.
• James Iry knows how to give description of Monads, and their usage: One Div Zero
• David MacIver has interesting theoretical insights to Scala: Desperately Unenterprise
• My own blog Scalada, and my example site Scalad
• Rumination's of a Programmer has lots of examples how to use Scala by Debasish Ghosh.
• Alex Blewitt's introductions for beginners
• Daniel Spiewak's Scala for Java Refugees articles
• Scala Buzz contains lots of Scala related weblogs
  
• Scala wiki has examples, and answers to frequently asked questions.
• Scala Blog - 'nuff said.
  

If anyone knows other good ones, I'd be happy to hear about them.

Now you know some of the people dealing with Scala. Next I'll give you some pointers to faciliation tools:

One of the most useful tools is a simple seach facility for Scala API. Even though it could be so much more, it's already good as is: API Seach tool. And for those who need to show short samples of code to others, e.g. in IRC discussion, a pastebin is very useful, and this one can even colorize the Scala syntax: Scala Pastebin.

Moreover, even though Scala standard library is very important, there's huge amount of useful general tools a programmer needs that can't be found from the standard library. Of course there's also Java API that can be utilized, but it doesn't let use all Scala features smoothly unless wrappers are explicitly written. That's why some clever guys started a community based library, called Scalax. I emphasize it's in its early phase, and encourage you to just wait until it's in better shape, unless you're enthusiastic and want to contribute to it. Moreover, there's another non-related publicly available Open Source code project named Scalaz, which somewhat overlaps with Scalax in its functionality. The design is however different, and you can get the better of both worlds if needed.

Of course I shouldn't forget the best open source real world example Scala has to offer yet: Lift webframework. There's also Scala-tools.org where Scala Open Source projects can be added, as far as I know.

Text editing tools, including IDEs, are numerous. For lightweight editors you can find a Scala configure file in the Scala distribution in [Scala]/share/scala/support/. IDEs are at the moment in development, but some are usable, for example Eclipse plugin which will have its newest, highly improved version out soon (update: it's in public testing now, see wiki. You can also read the progress of Netbeans plugin here.

I hope these descriptions helped you to get a better understanding where to go in the Scala world. By writing articles like this I try to give my humble support to this community, which I so enjoy being part of.

Roman numerals in Scala

I just read about converting Ints to Romans numerals in Python and Haskell and thought... gee... I can do that in Scala:



def romanize(number: Int) = {
  val numerals = List(("M", 1000), ("CM", 900), ("D", 500), ("CD", 400),
        ("C", 100),  ("XC", 90),  ("L", 50),  ("XL", 40),
        ("X", 10),   ("IX", 9),   ("V", 5),   ("IV", 4),
        ("I", 1))

  def next(in: Int) = numerals.filter(_._2 <= in) match {
    case (s, v) :: _ => Some((s, in - v))
    case _ => None
  }
  
  unfold(number)(next).mkString("")
}

def unfold[T, R](init: T)(f: T => Option[(R, T)]): List[R] = f(init) match {
  case None => Nil
  case Some(r, v) => r :: unfold(v)(f)
}




The 'unfold' function is not part of the standard Scala distribution, but looks to be darned useful. It takes an initial value and a function that returns the "next" thing and unrolls the initial value into a List. Nifty.

The actual "romanize" function contains a local definition of the roman numerals and the "next" function which finds the next match in the list of roman numerals. This function, along with the initial value are passed into unfold which returns a List of Roman numerals. mkString turns the list in a single String.

Short, sweet, and to my eye, the Scala and Haskell versions are very similar in complexity.

Announcing lift 0.4

Folks,

The lift team is pleased to announce the release of version 0.4 of the lift web framework.

lift is an expressive and elegant open source framework for writing web applications. lift stresses the importance of security, maintainability, scalability and performance, while allowing for high levels of developer productivity.

lift borrows from the best of existing frameworks, providing

* Seaside's highly granular sessions and security
* Rails fast flash-to-bang
* Django's "more than just CRUD is included"
* Wicket's designer-friendly templating style (see Lift View First)

And because lift applications are written in Scala, an elegant new JVM language, you can still use your favorite Java libraries and deploy to your favorite Servlet Container. Use the code you've already written and deploy to the container you've already configured!

Version 0.4 marks a few major milestones for lift. It's the first time-based release we've done for lift. We're planning to do a monthly release between now and the 1.0 release of lift.

Version 0.4 also marks the first time folks other than me have made the largest contributions to lift during a release. Maciek's PostgreSQL support and DavidB's <head> rewrite contributions are amazing and important to lift. SteveJ continues his messaging rampage with XMPP support. Additionally, DavidB and Eric Torreborre (who has just joined the lift team) have been working on a test framework for lift.

So, please try lift 0.4, join the lift community and give us some feedback.

Thanks,

David

Changes in this version include:

New features:
o Added XMPP support for lift applications. see the package net.liftweb.xmpp
o Added support for switching XHTML MIME type serving off. See LiftServet.useXhtmlMimeType
o Adding support to H2 to lift-ORM
o Adding support to PostgreSQL to lift-ORM
o /html/body//head are merged into /html/head, to allow html fragment inlined into body to contribute to /html/head
o Adding a new lift archetype, lift-archetype-basic that will generate a lift template with basic database and users functionality.
o Adding LiftConsole to archetype to play with lift-based application in console mode
o Adding mock for HTTPRequest and Co
o Adding first Specs test to test lift-core


Changes:
o Upgrade to scala-2.6.1
o Support for PosgreSQL 8.1 and some tests
o The look of the maven site generated (for lift itself and for archetypes)

Removed:
o Removing deprecated new_proj (replaced by archetypes)
o Removing lift-archetype-hellolift (replaced by lift-archetype-basic (and the sample application hellolift))

maven for scala

I'll try to provide a mini-guide about maven for scala's project. For more info, go to maven home or plugin's home.

Introduction to maven

Maven is a builder like make or ant, written in java. It's a commande line tool, IDE (Eclipse, Netbeans, IDEA) have plugins to handle and integrate project powered by maven. It could be used to create lib (jar), webapps (ear) and any other type of "artifact". It prefers convention over configuration, and configuration over instruction. What that mean exactly ?

• every action have a default configuration (= the convention).
• every action is a goal defined into a plugin (aka mojo), and for common case, you will try to use existing plugin instead of calling (more) low level instruction (like copy file,...)

Before creating your first scala project, You need to know some info:

a command line tool

"mvn" is the name of the command line tool to call maven 2.x. To display help, run mvn help

the project descriptor : the file [prj]/pom.xml

It's the file when every project information are stored (name, version, dependencies, license, mailing-list,...)

the build lifecycle :

The build lifecycle is defined by a sequence of phases, the main are :

• compile - compile the source code of the project
• test - test the compiled source code using a suitable unit testing framework. These tests should not require the code be packaged or deployed
• package - take the compiled code and package it in its distributable format, such as a JAR.
• integration-test - process and deploy the package if necessary into an environment where integration tests can be run
• install - install the package into the local repository, for use as a dependency in other projects locally
• deploy - done in an integration or release environment, copies the final package to the remote repository for sharing with other developers and projects.

A phase is depend of the previous one, so when you request the phase test, then the phase compile is done before,...

Directory layout

see below for a scala project

repository

Maven use repositories (local and remote) to store and to retrieve artifacts and their descriptor (pom). Artifacts are jar, war,... and they could be used as dependencies, maven plugin,...

By default, maven search artifact in the central repository. A "dedicated" repository for Scala stuff is available at http://scala-tools.org/repo-releases/.

Your first scala project with maven

In the following, we will run maven for the first time. Maven download what it need to work from remote repositories, and cache the downloaded artifact into its local repository (default is $HOME/.m2/repository). It only download what it need for the requested phases/goals (lazy downloading). So the first runs could be very long.

Step 0: installation

• install jdk 1.5+ (eg : on my box $HOME/bin/soft-linux/jdk-1.5.0_03)
• install maven 2.0.8+ (eg : on my box $HOME/bin/soft-java/apache-maven-2.0.8)
  • download it
  • unarchive it
  • add the apache-maven-2.0.8/bin directory to your PATH
  • check that maven is in the path:
    • go into any directory outside the maven installation
    • run mvn help, you should see

      usage: mvn [options] [<goal(s)>] [<phase(s)>]
      
      Options:
      -q,--quiet                    Quiet output - only show errors
      ...
      

Step 1: create a project

You could create a project skeleton with your favorite file system tools (following directory layout as below) or you could use archetypes. Maven Archetypes are project'skeleton that could be used to create new project.

mvn org.apache.maven.plugins:maven-archetype-plugin:1.0-alpha-7:create \
-DarchetypeGroupId=org.scala-tools.archetypes \
-DarchetypeArtifactId=scala-archetype-simple \
-DarchetypeVersion=1.1 \
-DremoteRepositories=http://scala-tools.org/repo-releases \
-DgroupId=your.proj.gid -DartifactId=your-proj-id

At the end of the process you should see something like

...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1 second
[INFO] Finished at: Sat Jan 05 17:39:47 CET 2008
[INFO] Final Memory: 6M/63M
[INFO] ------------------------------------------------------------------------

!! Success you now have a empty project under your-proj-id directory with the following directory layout :

your-proj-id/
|-- pom.xml
`-- src
|-- main
|   `-- scala
|       `-- your
|           `-- proj
|               `-- gid
|                   `-- App.scala
`-- test
    `-- scala
        `-- your
            `-- proj
                `-- gid
                    `-- AppTest.scala

In fact, the project is not empty it contains an helloworld application (App.scala) and a JUnit test (AppTest.scala).
In the next step, you will request phase (or goals). The results will be put under your-proj-id/target directory. The target directory is the working directory where every plugin put the result of computation. If you want to clean up, request the goal "clean"

mvn clean

Step 2: compile the project

# only compile
mvn compile

If it's the first time you use maven with scala, the build should failed with a message like

...
[ERROR] FATAL ERROR
[INFO]
------------------------------------------------------------------------
[INFO] The PluginDescriptor for the plugin Plugin [org.scala-tools:maven-scala-plugin] was not found.
[INFO]
...

Cause :

the pom.xml (autogenerated) doesn't specify wish version to use for maven-scala-plugin, so maven try to use the latest available localy, and none was previously downloaded.

Solutions :

• edit the pom.xml and define a version for the plugin


• request to download the latest available on remote repositories

I prefer the second solution (in this case):

# only compile
mvn -U compile

now you should see

...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------
...

Step 3: compile and running test

The skeleton create a JUnit test AppTest.scala as sample, try to compile and run it

# compile + compile test + run test
mvn test

you should get :

...
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running your.proj.gid.AppTest
Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.054 sec <<< FAILURE!

Results :

Failed tests:
testKO(your.proj.gid.AppTest)

Tests run: 2, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] There are test failures.

Please refer to /home/dwayne/tmp/your-proj-id/target/surefire-reports for the individual test results.

BUILD FAILURE, it's not good! So read the log on console :

• there is 2 tests and one of them failed
• the failed test is the method testKO from the class your.proj.gid.AppTest
• see the content of the directory .../your-proj-id/target/surefire-reports for details

So you could read the problem in .../your-proj-id/target/surefire-reports/your.proj.gid.AppTest.txt

...
testKO(your.proj.gid.AppTest)  Time elapsed: 0.01 sec  <<< FAILURE!
junit.framework.AssertionFailedError
    at junit.framework.Assert.fail(Assert.java:47)
    at junit.framework.Assert.assertTrue(Assert.java:20)
    at junit.framework.Assert.assertTrue(Assert.java:27)
    at your.proj.gid.AppTest.testKO(AppTest.scala:26)
    at your.proj.gid.AppTest.testKO(AppTest.scala:26)
...

So edit the test and fix it (it's easy), and rerun test until it pass.
Why the empty project is created with a failed test? to check that test are running and are used.

Step 4: generate the jar

# compile + run test + generate the jar
mvn package

If you fixed the test in Step 3, then a jar should be generated under the target directory. The jar doesn't contains the test classes, only the classes from src/main/scala/...

Step 5: start coding

• add scala file under src/main/scala/... or src/test/scala/...
• run the phases or goal you wish,...
• if you need more lib (dependencies), edit the pom.xml and add <dependecy> node. By default you could declare dependency available on central repo (I suggest to use mvnrepository as a search engine in central repo), or in http://scala-tools.org/repo-releases/ (browse the directory, no search engine available :()

Conclusion

I expect this overlook and quick tutorial could help you to start playing with maven and scala. I plan to write other articles about "maven for scala" (about, the pom.xml and repositories). If you want to know more about maven and don't want to wai futur article, I suggest you browse the documentation of maven. I also suggest you to take a look at the maven-scala-plugin 2.x documentation, you'll see how to generate scaladoc, choose the scala version, or run a scala console with the project dependencies.

If you have question ask, If you want I detail some point here or in a futur article, ask.

Fun with Project Euler and Scala

Project Euler is a site with mathematical puzzles that can be solved relatively quickly with an efficient algorithm. Randall Munroe (of xkcd fame), used Project Euler to become more acquainted with Python, so I thought I'd give it a shot in Scala. I'll be using the Scala interpreter, which comes with any standard Scala installation.

Familiarity with Scala is not necessarily required, as I've attempted to also make this a whirlwind tour of some of Scala's more interesting features: implicit conversions, anonymous functions, anonymous parameters, type inference, lazy evaluation, Pimped Libraries, and sequence comprehensions. I also go through some of the methods in Scala's standard library, particularly for some of the collection classes. I don't attempt to explain all these things in detail, but I've provided links for the curious.

Warning: Puzzle spoilers ahead. Also, these algorithms are by no means optimal, but they're "fast enough" and they get the job done.

Problem 1: Find the sum of all the multiples of 3 or 5 below 1000.

This is fairly easy. The until method (defined on Ints via an implicit conversion to RichInt) lets us construct ranges of numbers.

scala> 1 until 10
res5: Range = Range(1, 2, 3, 4, 5, 6, 7, 8, 9)

We can use the filter method to grab only those numbers that are multiples of 3 or 5.

scala> (1 until 10).filter(n => n % 3 == 0 || n % 5 == 0)
res6: Seq.Projection[Int] = RangeF(3, 5, 6, 9)

We're using Scala's syntax for anonymous functions. If you're unfamiliar with Scala, it might seem like "n" is dynamically typed. In fact, Scala's type inference allows "n" to be statically typed (as an Int) at compile time, even though we omitted the type declaration.

And finally we can use foldLeft to sum up all the numbers.

scala> (1 until 1000).filter(n => n % 3 == 0 || n % 5 == 0).foldLeft(0)(_ + _)
res2: Int = 233168

Here the underscores in (_ + _) act as anonymous parameters for an anonymous function. The first underscore represents the first parameter, and the second underscore represents the second parameter. Pretty cool!

Problem 2: Find the sum of all the even-valued terms in the Fibonacci sequence which do not exceed one million.

First we need to define the Fibonacci sequence. We'll define it lazily using Scala's "lazy" construct and the Stream class.

scala> lazy val fib: Stream[Int] = Stream.cons(0,
     | Stream.cons(1, fib.zip(fib.tail).map(p => p._1 + p._2)))
fib: Stream[Int] = Stream(0, ?)

This manually defines the first two terms of the Fibonacci sequence, then recursively defines an infinite stream of the remaining Fibonacci terms. fib is the Fibonacci sequence starting at zero (0, 1, 1, 2, 3, ...). fib.tail is the Fibonacci sequence starting at one (1, 1, 2, 3, 5, ...). fib.zip(fib.tail) is the two sequences zipped into a sequence of pairs ((0, 1), (1, 1), (1, 2), (2, 3), ...). We then use map to sum the two parts of each pair (._1 and ._2) and complete the recursive definition of the rest of fib, after the first two terms (1, 2, 3, 5, ...).

Thanks to Stream, the terms of the Fibonacci sequence are only evaluated as they are needed, so we can represent an infinite stream without incurring infinite computation.

We can verify that we computed the Fibonacci numbers correctly by inspecting the first few terms of our Stream with take and print.

scala> fib.take(15).print
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, Stream.empty

That looks about right. Now lets filter so we only have the even-valued Fibonacci terms, use takeWhile to grab the terms below a million, and add them up. Since I'm getting tired of foldLeft, let's Pimp my Library and add a "sum" method to our Fibonacci sequence (indeed, to any Iterable[Int]).

scala> implicit def iterableWithSum(it: Iterable[Int]) =
     | new { def sum = it.foldLeft(0)(_ + _) }
iterableWithSum: (Iterable[Int])java.lang.Object{def sum: Int}

scala> fib.filter(_ % 2 == 0).takeWhile(_ <= 1000000).sum
res8: Int = 1089154

Problem 3: What is the largest prime factor of the number 317584931803?

We could define an infinite stream of prime numbers using the Sieve of Eratosthenes or some other technique for finding prime numbers, but this example is simple enough that we don't have to bother.

Let's recursively define an infinite stream of natural numbers and verify that it works as we intend.

scala> lazy val naturals: Stream[Int] = Stream.cons(1, naturals.map(_ + 1))
naturals: Stream[Int] = Stream(1, ?)

scala> naturals.take(10).print
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, Stream.empty

If we define the number we want to factor as a mutable var, we can use a combination of functional and imperative programming (fairly unique to Scala) to find the largest prime factor.

scala> var theNum = 317584931803L
theNum: Long = 317584931803

scala> naturals.drop(1).dropWhile(n => {while(theNum % n == 0) {theNum /= n}; theNum > 1})
res0: Stream[Int] = Stream(3919, ?)

We keep dividing down theNum until it has no factors left, then we return the last (largest) factor we used. By definition all the factors we divide by will be prime.

scala> naturals.dropWhile(n => if (theNum % n != 0) true else {theNum /= n; theNum > 1})

If a natural number is not a divisor of theNum, we drop it. If it is a divisor, we divide-and-update theNum, and drop it if theNum is still greater than 1. By updating theNum, we assure that all the divisors we find are prime. By stopping when theNum reaches 1, then the first number in our remaining stream will be the largest prime factor we want.

Problem 4: Find the largest palindrome made from the product of two 3-digit numbers.

We'll need to check for palindromes, so this could come in handy:

scala> def isPalindrome(s: String): Boolean = s.reverse.mkString == s
isPalindrome: (String)Boolean

scala> isPalindrome("1001")
res46: Boolean = true

We can use sequence comprehensions to generate all the products of three digit numbers that are palindromes.

scala> val palindromes = for (a <- (100 until 1000);
     | b <- (a until 1000);
     | val p = a*b if isPalindrome(p.toString))
     | yield p
palindromes: Seq.Projection[Int] = RangeG(10201, 11211, 12221, 13231...

Now we can sort and grab the largest palindrome with head.

scala> palindromes.toList.sort(_ > _).head
res52: Int = 906609

Problem 5: What is the smallest number that is evenly divisible by all of the numbers from 1 to 20?

I'm going to cheat here, because this is easier to do by hand:

scala> 2*3*2*5*7*2*3*11*13*2*17*19
res0: Int = 232792560

And that's a little fun with Project Euler and Scala!

I love Scala Actors

One of my favorite things about Scala is it's Actor library.

Messaging has become an important part of modern architecture for the web. Many of the large scale web sites that are popular today are using message queues as infrastructure for processing requests. Scala Actors help us use those ideas in lift. Recently we added AMQP support to lift by wrapping RabbitMQ's client libraries with Scala Actors. Scala Actors make it easy to model the consumer/producer paradigm that's so prevalent in modern messaging systems.

Another thing I love about Scala Actors is combining them with my favorite functional combinators to make useful data structures. Recently, for lift, I wrote some example code that I felt could benefit from a cache. 30 lines of Actors and functional combinators later and I had a full-featured cache where listeners could register and be updated when the state of the cache changed.

Actors are a wonderful tool for software design. If you want to learn more about them, I recommend reading the Scala Actors papers.

Regexp but no state machine

Let me tell you a story on pure functional programming and the Scala language.

Functional programming led to amazingly beautiful things like the
Curry-Howard correspondence, MapReduce and the lift web framework.

I want to show you a simple program that matches regular expressions (regexps), which demonstrates rather nicely what makes functional programming so appealing. You probably know what regexps are and that they can be turned to state machines and so on...

Let's forget about that for a moment. Let us start from scratch.

Well, not entirely from scratch. Speaking the words of theory, we say, a *word* w is either the empty word, a single *letter* c, or the concatenation of two words w₁,w₂. A regular expression then describes a set of words that all have a particular structure in common.

In order to write down regexps in Scala, we need to define the building blocks. And since we would like to keep the door open to matching other things than strings of characters, these building blocks should be generic. Here is the code.

abstract class RegExp[A]

case class Phi[A]() extends RegExp[A]
case class Empty[A]() extends RegExp[A]
case class L[A](letter:A) extends RegExp[A]
case class Choice[A](left:RegExp[A], right:RegExp[A]) extends RegExp[A]
case class Seq[A](left:RegExp[A], right:RegExp[A]) extends RegExp[A]
case class Star[A](exp:RegExp[A]) extends RegExp[A]

We can now directly write things like Choice(L('a'),Star(L('b'))). Here is what these expressions mean:

Phi matches nothing
Empty matches the empty word
L(c) matches the single letter c
Seq(r1,r2) match all words w₁,w₂ where w₁ matches r₁ and w₂ matches r₂
Choice(r1,r2) matches all words w that match either r₁ or r₂ (or both)
Star(r) matches all words w₁,...,w_n where each of the w_i matches r.

Now how can we write code that follows these definitions? Our first try might be a recursive function accepts that takes a regexp and a word and returns a boolean. Whenever a regexp contains another regexp, we could then call accept recursively, with suitable new input. Most rules seems easy, like for Choice we can try to match the input against r₁ and if that doesn't work out, we would try r₂. However, for Seq and Star we would have to cut a word in pieces and match the pieces against the regexps, possibly backtracking and trying out other ways to cut the word in pieces. That is quite inefficient.

It turns out there is a better way: A word can obviously only match if each of its letters were matched by some part of the regular expression. So why don't we take each letter, and try to "consume" it with some part of the regular expression. If we can play this game until no letters are left, this means the word is accepted. The key to making this work is to represent the "remaining" expression. But hey, don't we have everything there to represent expressions? We could just build the remaining expression as we go.

Here is the full program. Check out the partDeriv function, which computes the 'derivative' of a regular expression with respect to an input letter.

object Main {
  abstract class RegExp[A]
  case class Phi[A]() extends RegExp[A]
  case class Empty[A]() extends RegExp[A]
  case class L[A](letter:A) extends RegExp[A]
  case class Choice[A](left:RegExp[A], right:RegExp[A]) extends RegExp[A]
  case class Seq[A](left:RegExp[A], right:RegExp[A]) extends RegExp[A]
  case class Star[A](exp:RegExp[A]) extends RegExp[A]

  type Word[A] = List[A]

  def matchRegExp[A](re:RegExp[A], w:Word[A]): Boolean =
    w match {
      case List() => acceptsEmpty(re)
      case c::w1  => matchRegExp(partDeriv(re, c), w1)
    }

  def acceptsEmpty[A](re:RegExp[A]): Boolean =
    re match {      
      case Phi() => false
      case Empty() => true
      case Choice(r1,r2) => acceptsEmpty(r1) || acceptsEmpty(r2)      
      case Seq(r1,r2) => acceptsEmpty(r1) && acceptsEmpty(r2)
      case Star(r) => true
      case L(_) => false
    }

  def partDeriv[A](re:RegExp[A], c:A): RegExp[A] =
    (re, c) match {
      case (Phi(), _)                => Phi()
      case (Empty(), _)              => Phi()
      case (L(c1), c2) if (c1 == c2) => Empty()
      case (L(c1), _)                => Phi()
      case (Choice(r1,r2), c)        => Choice(partDeriv(r1,c),partDeriv(r2,c))
      case (Seq(r1,r2), c)           =>
        val rn = Seq(partDeriv(r1,c),r2)
        if (acceptsEmpty(r1)) Choice(rn, partDeriv(r2,c)) else rn
      case (rs @ Star(r), c)         => Seq(partDeriv(r,c),rs)
    }

  def main(args:Array[String]): Unit = {
    println matchRegExp(Choice(L('a'),Star(L('b'))), List('b','b','b'))
  }
}

What do you know? Pattern matching and recursive calls. Not a single imperative update. It doesn't even use anonymous functions. The program looks remarkably similar in Haskell.

I could now go and try to explain why the partDeriv function does what it does, but you wouldn't learn anything from it. There are two things that are helpful to understand why this works: one is calculating what the function does "by hand", using sample inputs (feel free to insert debug print statements and run it if you are too lazy for that). And, if that is not enough, read the theorem and proof on the 1964 paper by Janus A. Brzozowski which is called "Derivatives of Regular Expressions" JACM 11:4 pp 481-494.

What it comes down to is that some computational problems have a structure to them that permits elegant solutions. David Pollak has summed it up nicely when he said about Scala "what it does is, it reduces code to its essence". Not every problem will have an elegant purely functional solution, but conciseness and clarity have their place in every software engineering project. Now enjoy finding the essence of your code.

Dynamic web applications with lift and Scala

This is a tutorial introduction to building dynamic web applications with lift and Scala. We'll be building a linksharing site, much like Digg or reddit. However, lift's advanced support for Comet-enabled web applications will allow our site to update live as new links get submitted or old links get voted up or down, without our visitors having to refresh the page.



This is a short (50 seconds) screencast of what we're trying to build. The quality on the YouTube version isn't great, so you can also download the full-quality QuickTime version (~800K). I'm running two different browsers, Safari and Camino, and loading the finished site with both of them. Notice how the link submissions and votes made in one browser get automatically updated in the other browser. We accomplish this using Comet, and we'll see how lift makes Comet-enabled web applications a breeze. Let's get started.

I'll assume that you have the most recent versions of Scala, Eclipse, the Scala Plugin for Eclipse, and Maven installed. I'll also assume you have some rudimentary knowledge of Scala.

(I've you have used lift before the 0.3.0 release on Dec. 7, there is a bug in our repository that might cause you to use a pre-release version of the lift libraries. To fix this problem, delete the net/liftweb directory in your local Maven repository before continuing. I believe the default location is: ~/.m2/repository/net/liftweb)

From Eclipse's workspace directory, lets create a new project called "linkshare" in the "com.test" package by typing the following into the command line:

mvn org.apache.maven.plugins:maven-archetype-plugin:1.0-alpha-7:create -U \
-DarchetypeGroupId=net.liftweb                             \
-DarchetypeArtifactId=lift-archetype-blank                 \
-DarchetypeVersion=0.3.0                                   \
-DremoteRepositories=http://scala-tools.org/repo-releases  \
-DgroupId=com.test -DartifactId=linkshare

That's all we need to do to set up a skeleton lift site. (If you're using Eclipse and want to use Q4E, see: http://liftweb.net/index.php/Archetypes) Now we'll go into the directory that contains our project and run it with Jetty. Maven should automatically download Jetty for you if you don't already have it.

cd linkshare
mvn jetty:run

If everything goes well, you should be able to see your skeleton lift site running on http://localhost:8080/. We see a nice welcome screen that shows the current time. Now let's do something with the site. Leave Jetty running, and open the index.html file in src/main/webapp. Edit the file so that it contains a more topical message, like "Welcome to your linksharing site!". Save the file and reload your browser. Your changes should have been picked up by Jetty.

Jetty also automatically picks up on changes to your compiled class files and redeploys your webapp when necessary, so you can leave the Jetty process running throughout this tutorial and see the changes in your browser. We'll still want to do some stuff on the command line, so open up another command line window. Then create an eclipse project with the following command:

mvn eclipse:eclipse

Now you can import the lift project into Eclipse. There are two directories you should mostly be concerned with. The src/main/scala directory contains all the Scala code for your lift project, while the src/main/webapp directory contains the html, css, javascript, etc.

You'll probably want to enable Eclipse's incremental compilation. That way, whenever you save changes to your code, Eclipse will recompile your .class files, and Jetty will pick up the new code. If you're not using Eclipse, or not using incremental compilation, you'll want to issue a 'mvn compile' command to see any changes to your code reflected in Jetty.

You'll notice that our app's page has a "Home" link which points right back to the main page. This is actually a part of lift's advanced Sitemap feature, but for now we won't be using it, so let's disable it.

The Sitemap is defined in our bootstrap. Our site's bootstrap code gets run by lift as soon as our app gets deployed. This is a great place to make sure we set up our databases correctly, among other things. It's also where our Sitemap is defined, so let's go take a look.

You'll find the bootstrap code in src/main/scala/bootstrap/liftweb/Boot.scala. It's fairly lonely in here, as our app doesn't do much yet. Let's modify the appropriate line to hide the "Home" link from our Sitemap. We do this by adding the "Hidden" argument, like so:

    val entries = Menu(Loc("Home", "/", "Home", Hidden)) :: Nil

Save the file, then wait a few seconds for Jetty to pick up the changes. (If you're not using Eclipse's incremental compilation, then you'll have to issue a 'mvn install' command.) When you refresh your site's page, you should see that the "Home" link has disappeared. Awesome! Now let's do something useful.

Our first order of business is to allow our visitors to submit links. If we want to receive links from our visitors, then we also want somewhere to keep those links. At this point, most people would start defining database connections, database schemata, model objects, object-relational mapping... Doing all of this is very easy with lift. The included Mapper classes provide a great way of easily, securely, and meaningfully storing your bytes.

However, I'm going to make a controversial decision: we won't use a database. lift, you see, is a highly stateful web framework. It keeps state around between requests, so we can get very far without a database. Of course, this means that if our app gets redeployed, or if it goes down for maintenance, or if it crashes, then we'll lose all of our state and we'll have to start collecting links from scratch. This is clearly unacceptable in a production environment, where we would want to persist our important information to a database. Like I said, lift makes this easy with the included Mapper classes. However, for the purposes of this demo, we can deal with losing all of our links every now and then.

So if we're not keeping our links in a database, then where are we keeping them? We're going to use a long-lived Scala Actor. Scala Actors are lightweight threads, similar to Erlang's Actors. They can send and receive messages to other Actors, which will come in very handy. Let's make a LinkStore object in our com.test.controller package:

package com.test.controller;

import scala.collection.mutable.{HashMap, HashSet}
import scala.actors.Actor
import scala.actors.Actor._

import net.liftweb.util.Helpers._

// Messages
case class AddListener(listener: Actor)
case class RemoveListener(listener: Actor)
case class UpdateLinks(topLinks: List[Link])
case class AddLink(url: String, title: String)
case class VoteUp(linkId: String)
case class VoteDown(linkId: String)

// Data Structures
case class Link(id: String, entry: LinkEntry)
case class LinkEntry(url: String, title: String, var score: Int)

object LinkStore extends Actor {
val linkMap = new HashMap[String, LinkEntry]
val listeners = new HashSet[Actor]
var topLinks: List[Link] = Nil

def notifyListeners = {
  topLinks = linkMap.toList.map(p => Link(p._1, p._2)).
    sort(_.entry.score > _.entry.score).take(20)

  listeners.foreach(_ ! UpdateLinks(topLinks))
}

def act = {
  loop {
    react {
      case AddListener(listener: Actor) =>
        listeners.incl(listener)
        reply(UpdateLinks(topLinks))
      case RemoveListener(listener: Actor) =>
        listeners.excl(listener)
      case AddLink(url: String, title: String) =>
        linkMap += randomString(12) -> LinkEntry(url, title, 1)        
        notifyListeners
      case VoteUp(linkId: String) => try {
        linkMap(linkId).score += 1
        notifyListeners
      }
      case VoteDown(linkId: String) => try {
        linkMap(linkId).score -= 1
        notifyListeners
      }
    }
  }
}

start  // This starts our singleton Actor
}

The central data structure here is the LinkEntry. It stores the url, title, and current score of a link. Our LinkStore is a singleton Actor which is started up when the class is loaded. The LinkStore keeps a HashMap of LinkEntries, indexed by a unique ID of 12 characters. The LinkStore responds to three messages that can modify the LinkEntries: AddLink, VoteUp, and VoteDown. Each does what its name suggests. The LinkStore also keeps a HashSet of listeners. Listeners are other actors that want to be notified when the LinkStore changes. Registered listeners are sent the newest list of Top 20 Links whenever there is an update.

Now that we have a place to store our links, we want to let our visitors send them to us. If you take a look at index.html in src/main/webapp, you'll notice several <lift:...> tags. When a user requests a page, lift will try to find it in your src/main/webapp folder. If the page has any <lift:...> tags, they'll get processed before the response is sent back to the user. The first such tag we see is <lift:surround>. This tells lift to surround the enclosed xhtml with the "default" template. Templates can be found in src/main/webapp/templates-hidden. The second such tag is a <lift:snippet> tag. This tag instructs lift to call the "helloWorld:howdy" snippet, and replace the <lift:snippet> tag with the results of the snippet call. Snippets can be found in src/main/scala/com/test/snippet. Let's take a look at the HelloWorld.scala snippet. HelloWorld is a class which defines one method, "howdy". All that howdy does is return some xhtml. This is the xhtml that will replace the <lift:snippet> tag before the whole page gets sent back to the user. Notice that Scala allows XML literals right in the syntax. It also allows arbitrary Scala expressions inside the XML when escaped by {} brackets. As you can imagine, embedding xhtml straight into your code can be very useful for web development.

Let's get rid of the HelloWorld snippet and replace it with our own. Make a new class in com.test.snippet called "Submission", and define a method "form":

package com.test.snippet;

import net.liftweb.http.S
import net.liftweb.util.Helpers._

import com.test.controller._

class Submission {
def form = {
  var url = ""
  var title = ""

  <span>
    { S.text("URL", u => url = u) }
    { S.text("Title", t => title = t) }
    { S.submit("Submit", ignore => LinkStore ! AddLink(url, title)) }
  </span>
}
}

Phew! There's a lot going on in these few lines. Let's try to break it down piece by piece. We're taking full-advantage of Scala's support for XML in the syntax to send back an xhtml form. We're using {} brackets to escape the XML syntax and insert arbitrary Scala expressions. The first and second Scala expressions reference "S.text". This is a method that will return a text input field. The first argument to the method is the default text in the field. The second argument is a function that will be executed on the contents of the text input field when the form is submitted. Take a moment to let that sink in. So when the form gets submitted, the stuff in the url field will be assigned to the "url" variable and the stuff in the title field will be assigned to the "title" variable.

"But how?!", you might ask. After all, Submission's "form" method gets called in one request-response cycle, and the html form doesn't get submitted until the next request-response cycle! Well, this is part of lift's magic. Between requests, lift keeps around a closure that knows about the "url" variable and about the "title" variable. Even better, the closure knows what to do with these variables once the form submission comes in. It works almost like magic.

Which leaves us one last thing to analyze. As you might have guessed, "S.submit" returns a submit input field (a submit button). The first argument to "S.submit" is the label on the button. The second argument is a function to be called once the form is submitted. This function sends a message (using "!", a method defined on Actors) to the LinkStore, telling it that we want to add a new link.

Now let's go back to index.html and change the "helloWorld:howdy" snippet to a "submission:form" snippet, like so:

    <p><lift:snippet type="submission:form" form="post" /></p>

Notice that we added a "form" attribute to let lift know that this snippet should be rendered as a POST form.

Reload your page and you should now see two form fields and a submission button. Entering information and pressing submit doesn't seem to do anything, but if we add a "Console.println" to LinkStore we'll notice that it is indeed receiving and storing links.

Now that we have links, how do we share them with the world? Something needs to be responsible for showing these links to our visitors and updating them whenever necessary. This sounds like just the job for lift's CometActor. CometActors know how to render themselves onto an xhtml page. They are also long-lived; a visitor can navigate away from the page and come back to find the same CometActor servicing his requests. Since they're long-lived, they can interact asynchronously with the visitor, outside of the normal http request/response cycle. We'll use these capabilities to make "live" updates to the page a visitor is viewing, without the visitor having to refresh the page. Let's get cranking. Add a new "com.test.comet" package, and in it define a new LinkActor class:

package com.test.comet;

import scala.collection.mutable.HashMap
import scala.xml.{NodeSeq, Text}

import net.liftweb.http._
import net.liftweb.http.js.JsCmds.Noop
import net.liftweb.util.Can

import com.test.controller._

class LinkActor(theSession: LiftSession, name: Can[String],
defaultXml: NodeSeq, attributes: Map[String, String])
extends CometActor(theSession, name, defaultXml, attributes) {
var topLinks: List[Link] = Nil

def defaultPrefix = "links"

def render = {
  def linkView(link: Link): NodeSeq = {
    <li>
      <a href={link.entry.url}>{link.entry.title}</a>
      [{link.entry.score} {if (link.entry.score == 1) "vote" else "votes"}]
      {S.a(() => {LinkStore ! VoteUp(link.id); Noop}, Text("[up]"))}
      {S.a(() => {LinkStore ! VoteDown(link.id); Noop}, Text("[down]"))}
    </li>
  }
    
  bind("view" -> <ol>{topLinks.flatMap(linkView _)}</ol>)
}

override def localSetup {
  LinkStore !? AddListener(this) match {
    case UpdateLinks(links) => topLinks = links
  }
}

override def localShutdown {
  LinkStore ! RemoveListener(this)
}

override def lowPriority : PartialFunction[Any, Unit] = {
  case UpdateLinks(newLinks) => topLinks = newLinks; reRender(false)
}
}

Let's start with the easy stuff. The "localSetup" method is called when the CometActor is created. This method sends a message to LinkStore (to make sure it gets added as a listener) and waits for a reply. The reply will contain the initial list of top links, which we store in the "topLinks" variable. The "localShutdown" method is called when the CometActor gets retired. This just cleans up and makes sure we're no longer listening to the LinkStore.

The "render" method does most of the interest work. This is how the CometActor renders itself to xhtml. It's going to create some xhtml with an ordered list of links. The links themselves get rendered by the inner "linkView" method. This method will render the link with the appropriate title, as well as showing the score beside the link, and "[up]" and "[down]" links to vote the story up or down. The "S.a" method takes two parameters, and returns the xhtml code for an anchor tag (link). The first parameter is the code that will be executed when the link is pressed. This code sends a message to the LinkStore, telling it whether the vote was an up-vote or a down-vote. (The "Noop" at the end of the block is just to let Scala know that this is indeed a JsCmd block that is safe to send to the browser.) The second parameter is an xhtml node that will be wrapped inside the anchor tag. In this case, it's just a simple text node that indicates whether the vote is up or down. With these parameters, lift creates an anchor tag that, when clicked, will send an AJAX request to the server to execute the provided code. Neat stuff.

Finally, the "lowPriority" method is where we listen for updates from the LinkStore. As you might imagine, CometActors have three similar methods: "lowPriority", "mediumPriority", and "highPriority". The only differences are the order in which they get applied. We're not doing anything too complicated with CometActors here, so "lowPriority" will work fine for our purposes. Whenever we get an UpdateLinks message from the LinkStore, we want to update our local copy of the links. We also call "reRender", which pushes the latest result of "render" to the page containing the corresponding <lift:comet> tag for this CometActor.

The last thing we have to do is place the <lift:comet> tag on a page. This will render the CometActor we've just created to that page, and will listen to the CometActor for updates. Let's go back to index.html and add the following right below our form submission:

    <p>
    <lift:comet type="LinkActor">
      <links:view>Loading...</links:view>
    </lift:comet>
  </p>

Now you should have a site that functions exactly like the site in the screencast. If you open two different browsers, or two different windows in the same browser, actions taken in one browser window will be automatically reflected in the second browser window.

This introduction barely touches the full power of lift. We didn't really explore lift's Sitemap, ORM Mapper classes, extensive type-safety, semantic models, security-by-default, or advanced templating system. Hopefully, however, this gives you a taste of web development with lift, and showcases just one of lift's many features: advanced support for Comet-style web development.

This code borrows heavily from Steve Jenson's DynamicBlog, a part of the "hellolift" example project. Check it out at the lift source tree (http://liftweb.googlecode.com/svn/trunk/). Thanks to David Bernard for his help with syntax highlighting and with his feedback on a draft version of this blog post. And many, many thanks to David Pollak for his feedback on this blog post, but most of all for making such an awesome web framework.

A year with Scala and lift

I was a bit more than a year ago that I stumbled across Scala.

I had been doing Ruby and Rails for 18 months and Java for 10 years before I found Scala.

I was looking for a language that had the strengths of both Java and Ruby and I found them in Scala. I have been happy with the language and the community ever since.

Scala's goodness comes in many forms. Some practical (XML as part of the language syntax, runs on the JVM and has 100% compatibility with Java libraries), some geeky (Scala's type system is powerful and learning to program with primarily immutable data types is an excellent thought-challenge), and esoteric (Scala has an implementation of Erlang's Actors that's pretty spectacular.)

After coding in Scala for year, I find it very difficult to code in Ruby and almost impossible to code in Java. Scala has spoiled me on a bunch of different vectors.

Scala has a command-line interpreter. I can type lines of Scala, have them run... just like IRB in Ruby. That makes exploring a new API much, much easier. For example, I was writing some Crypto stuff the other day (in Java). I was able to check out the Java crypto stuff, run simple examples, etc. from the Scala command line. Once I was satisfied with the code, I ported it to Java and ran it. It's a lot easier than a "make changes to code, run the new program, see what it did" cycle with Java.

Scala's got XML right in the language. I was doing some SOAP coding recently. I captured the wire-level of some existing SOAP calls, copied the XML into my Scala code, and I had a running SOAP client in about 30 minutes. Some of the folks who were doing the same coding task in Java didn't even have all the import statements defined in by the time I had my code running.

Scala allows passing of functions and storing functions in Hashes, etc. It's like having amazingly light weight anonymous inner classes all over your code. In reality, the Scala compiler is creating a bunch of anonymous inner classes, but your code isn't cluttered with all the class definition stuff... your code is a simple statement that looks and feels like it's part of the current method. From a "picking your code up standpoint" it's much easier to understand what you did if your eye sees the substance rather than the cruft that is anonymous class definition.

You can code Scala just like you code Java. All the Java constructs are there (except the c-style for loop, but you'll never miss it.) You can connect to your Java classes and Java libraries from Scala. You can subclass Java classes, implement Java methods and your Scala code looks just like Java code to the other libraries. It's very simple to stick your toe in the Scala water and see if you like the temperature. Also, Scala code runs as fast as Java code so you're not sacrificing performance for language value.

Scala's traits and type system allow for type-safe, compiler checked meta-style programming. No, it's not changing the behavior of classes at runtime, but it is the ability to add complex functionality to classes simply by mixing new methods in. It's far more powerful than Ruby's mixin mechanism and the compiler takes care of making sure things don't get broken. An example of this is adding "user" functionality to an OR mapped class (first name, last name, email, password, etc.) In Ruby, this would be done with acts_as_xxx, in Scala this is done with class Foo extends Mapper with ProtoUser. And, yes, you can, at compile time, add methods to existing classes, even String.

So... what about lift?

One of the things that drove me to search for a new language after doing Ruby was the multi-fold needs for performance, type safety, and integration with Java libraries including Lucene.

I found Scala, but there were no web frameworks for Scala. Sure, I could have coded in Wicket and Hibernate using Scala, but then I would have lost a bunch of the Scala benefits. So, I sat down to write my own web framework. I started with a list of requirements and started coding.

I initially built lift "for me." Later, I started building lift for a multi-person team that I was part of.

Finally, the lift community developed a life of its own and people started using lift for real projects. There are currently 300+ people in the lift community, 5 active committers, and a bunch of active projects being built on lift (including Much4).

There are some amazing people in the lift community... people who know so much more than I do about so many different things. There's great give and take on the lift mailing list.

Most importantly, we're moving lift towards being a 1.0 product... a product that you can build a commercial application on top of. This means being developer friendly, having good documentation, having a decent tool chain.

So... what's so cool about lift?

It's got the code efficiency of Rails -- a few lines gets you a lot of stuff.

It's got the designer friendliness of Wicket.

It's got the statefulness of Wicket and Seaside.

Plus, lift is the best way to write AJAX and Comet applications. With a dozen lines, you can put a dynamically updating clock in your application. You can add chat with 50 lines. All this is possible because of Scala's Actors and the way lift abstracts away the HTTP request/response cycle.

Anyway... stay tuned to the Scala Blogs to see exciting developments in Scala and lift brought to you by some of the coolest people in the Scala and lift communities.