[Fwd: Re: continue keyword]

Yeah, it returns a Seq[String] which is backed by a List[String].

And yeah, the for-comprehension is just a sequence of calls to filter, map, and flatMap, which preserve the laziness of ManagedSequence.

And yes, your revised version works.

--j

On Fri, Mar 20, 2009 at 3:08 PM, Brett Knights <brett [at] knightsofthenet [dot] com> wrote:

Jorge Ortiz wrote:

   Doesn't this one still read all the lines?

 No. The 'lines' method returns a ManagedSequence[String], which is a lazy "collection". Evaluation (and thus, file access) will be delayed until it's needed. Because of the takeWhile condition, evaluation (and thus, file access) will terminate early. Thanks to lazy evaluation, it also processes the file in O(1) (constant) memory.

Your code, on the other hand, will not only read the entire file, it will put the entire file into memory (O(N)), as readLines returns a (strict) List[String]. Not only will your code read the entire file, it will throw OOMException for large files.

--j

Actually I thought that I was using readLines from scalax that produces a Seq[String]. Oh well. It also works just as well to use lines so the following is revised:


      import scalax.io.Implicits._
      import java.io.File

      val infile = new File(args(0));

      var go = true;
      for { l <- infile.lines.takeWhile(l => go)}{
          l.trim() match{ // no need for continue
                  case "ted" => go = false; // break handled
                  case x => println(x);
              }
      }

but what I think you are saying is that your original for comprehension is actually being pulled from only so long as the takeWhile returns true. That's pretty cool if that is what's happening.

(for {l <- infile.toFile.lines
... guards etc
    } yield LineTime(line, time)).takeWhile

This is my best shot at recreating that in Scala. I'm assuming Scalax's IO library, which is lazy.

The "skip" conditions have to be turned into "keep" conditions, but that's just a matter of applying logical NOT to them.

  import scalax.data.Implicits._
  import scalax.io.Implicits._

  case class LineTime(line: String, time: Float)

  (for {
    l <- infile.toFile.lines
    val line = l.trim
    if (line != "" && !line.startsWith("#"))

    val data = line.split(",").toList
    if data.length >= 3

    val ac = data(2)
    if (!acpair.isDefined || acpair.contains(ac) || line.startsWith("WND "))

    val time = data(1).toFloat * sec
    if (time >= startTime || data(0) != "TRK")
  } yield LineTime(line, time)).takeWhile(_.time <= endTime).map(_.line)

Now, that last line is arguably a bit ugly. It'd be the only thing I'd change if break/continue were made available for Scala collections. I might make the code look like so:

  for {
    l <- infile.toFile.lines
    val line = l.trim
    if (line != "" && !line.startsWith("#"))

    val data = line.split(",").toList
    if data.length >= 3

    val ac = data(2)
    if (!acpair.isDefined || acpair.contains(ac) || line.startsWith("WND "))

    val time = data(1).toFloat * sec
    if (time >= startTime || data(0) != "TRK")
    val _ = if (time > endTime) break
  } yield line

--j

On Thu, Mar 19, 2009 at 12:37 PM, Russ Paielli <russ [dot] paielli [at] gmail [dot] com> wrote:

Detering Dirk wrote:
> I would be really, really interested in seeing a simplified
> (but not oversimplified!) fully functional prototype version
> of this usecase written in your style, and than let us
> (well, not me, but the experts ;) ) show how that would be
> solved in The Scala Way.
>
> It seems a sufficiently complex, but not too complex,
> properly isolated problem for a showcase, and even coming
> out of real world necessities.
>

OK, you asked for it, so here is a sample for you. This is a section from one of several Python scripts I use for processing the air traffic data files I mentioned. This is the simplest of those scripts, but it has the basics. Sorry, but this section of code by itself is not fully functional. First, here is the header comment just to give you a clue what it is doing:

"""
This Python script by Russ Paielli extracts data for specified flights
and a specified time window around an operational error from a TSAFE
input data file (default: TSAFE-all.in). It puts the data into another
smaller TSAFE input file (default: TSAFE.in). By default, the aircraft
IDs of the conflict pair are specified in the input file ACpair.dat. If
the --all option is used, all flights are extracted for the specified
time period.
"""

And here is the main looping portion of the script:

count = 0

for line in file(infile): # main loop

    line = line.strip()
    if not line or line.startswith("#"): print >> out, line; continue # skip blank line or comment

    data = line.split()
    if len(data) < 3: continue # bad data line
    AC = data[2] # aircraft ID

    if ACpair and AC not in ACpair and not line.startswith("WND "): continue

    time = float(data[1]) * sec

    if time < starttime and data[0] == "TRK": continue
    if time > endtime: break

    print >> out, line

    count += 1