Can’t do http.request? Are you behind the proxy?

December 20th, 2011

If that’s the case then instead of:

http.get({
    host: "www.google.com",
    path: "/",
    port: 80,
    method: "GET"
  }, function(res) {
  ...
});

do:

http.get({
    host: "proxy.url.com", // <- no protocol
    path: "http://www.google.com/#sclient=psy-ab&hl=en&safe=off&q=node.js&pbx=1",
    port: 8080, // <- your proxy port
    method: "GET",
    headers: {
      Host: "www.google.com"
    }
  }, function(res2) {
  ...
});

This will send your request via your proxy server.

e4x for ColdFusion, sort of…

May 17th, 2011

Coming from a Flex background I can’t imagine working with XML and not having e4x available. As you most likely know ColdFusion doesn’t have support for this fantastic notation. It has it’s own, clunky syntax for accessing XML elements within the document and an XmlSearch() function to query the document with XPath. I used to like XPath – until I saw e4x.

While I was waking up today I had this crazy idea – add e4x to CF. Here is what I came up with, a <cfx_e4x /> custom tag. Mind you, it’s only a couple of hours of work but it seems to be usable. I have tested this with CF9 on Java 6.

Basic syntax:

<cfscript>
 x = xmlParse("<rootTag><tag attr='1'>value 1</tag><tag attr='2' nextAttr='meh'>value 2</tag></rootTag>");
</cfscript>
<cfx_e4x xml="#x#" e4x="root.tag.(@attr == 2).@nextAttr" />
<cfoutput>#xmlParse(cfx_e4x).result.xmlText#</cfoutput>

The above code will print meh. Cool, but what exactly happens here?

Lets start with where root. in an e4x expression comes from. Behind the scenes this tag uses Rhino to execute an e4x expression on the XML data. To get that working the XML has to be assigned to the variable – this variable is called root. But you can control what this variable is called. If you wanted to call it lol your call would look:

...
<cfx_e4x xml="#x#" e4x="lol.tag.(@attr == 2).@nextAttr" e4xRoot="lol" />
<cfoutput>#xmlParse(cfx_e4x).result.xmlText#</cfoutput>

I’m sure you have noticed that the result of the tag was again parsed with xmlParse(). There is a very important reason for that. Depending of what the result of e4x expression is you may get a simple value, single or multiple XML elements in return. To make all these types of responses easily available cfx_e4x is wrapping the result into a top XML element. By default this element is called result. If your e4x was:

root.tag

the response in plain text would look like this:

<tag attr="1">value 1</tag>
<tag attr="2" nextAttr="meh">value 2</tag>

You wouldn’t be able to parse it easily without adding the root node yourself. cfx_e4x will return:

<result>
<tag attr="1">value 1</tag>
<tag attr="2" nextAttr="meh">value 2</tag>
</result>

Again, you can control the name of root node name using resultName attribute. If you execute the tag in a following way:

<cfx_e4x xml="#x#" e4x="lol.tag.(@attr == 2).@nextAttr" e4xRoot="lol" resultName="top" />

you would output the result using:

<cfoutput>#xmlParse(cfx_e4x).top.xmlText#</cfoutput>

Last to mention are namespaces. I found this XML on stackoverflow:

<rdf:RDF
 xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
 xmlns="http://purl.org/rss/1.0/">
    <rdf:item value="5"/>
    <item value="10"/>
</rdf:RDF>

What if you wanted to get the values attributes for rdf:item and item elements? You would have to manually tell cfx_e4x what are the namespaces. Take a look at the code:

<cfscript>
 x = xmlParse( "<rdf:RDF
  xmlns:rdf=""http://www.w3.org/1999/02/22-rdf-syntax-ns##""
  xmlns=""http://purl.org/rss/1.0/"">
   <rdf:item value=""5""/>
   <item value=""10""/>
  </rdf:RDF>"
 );
</cfscript>
<cfx_e4x xml="#x#" e4x="root.reg::item.@value"
 ns_rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns##"
 ns_reg="http://purl.org/rss/1.0/" />
<cfdump var="#xmlParse(cfx_e4x)#" />

By using ns_... attribute you can tell what namespaces are being used. The only one you have to watch out for is the default namesapce. You have to name it – here it is named reg.

cfx_e4x is just a proof of concept. The code is available on github under an MIT license.

To install and use, download latest Rhino version, unzip/untar and copy js.jar into {CFROOT}/lib directory. Download CFXe4x.jar from github or build from source and drop it in the same directory. Restart ColdFusion and register a custom tag with name cfx_e4x and class uk.co.appembassy.cf.e4x.CFXe4x.

AppEmbassy Ltd

May 16th, 2011

There’s a new boy in the town – AppEmbassy Ltd.

ColdFusion 9 crashing? Try different garbage collection

March 24th, 2011

For the ICS application we are using ColdFusion 9. For quite a while we had a major problem. Our CF server kept crashing at least once a day without any warning. One minute it was running just fine, 30 seconds later all memory was consumed (3GB assigned for the JVM). Server was totally becoming unresponsive and the only way to bring it back to life was restarting CF service.

FusionReactor was no help for us, nor memory snapshots. It looked like we were blind, couldn’t find the problem. The code was optimised as much as possible, we went through every query, created indexes on the databases, profiled it, var’d all variables in the CFCs and so on – nothing was helping. And then finally, Saturday morning that was, a breakthrough: there’s clearly a problem with garbage collection (how obvious was that…), how can we tune it? Turns out that JVM supports 3 different collectors: throughput (-XX:+UseParallelGC), concurrent (-XX:+UseConcMarkSweepGC) and incremental (-XX:+UseTrainGC) – here’s the details “Tuning Garbage Collection with the 5.0 Java[tm] Virtual Machine”. My first thought was that we may have problems because we’re running in a VM, each of those VMs has 4 virtual CPUs assigned. According to the JVM docs we should have 4 collector threads by default. Somehow, at some point they were all failing to collect. So I changed the collector to concurrent and since then, about 5 weeks ago we had no crashes at all.

This is what I’m using for JVM settings:

-server -Dsun.io.useCanonCaches=false -XX:MaxPermSize=192m -XX:+UseConcMarkSweepGC -Xbatch -Dcoldfusion.rootDir={application.home}/../ -Dcoldfusion.libPath={application.home}/../lib -XX:+HeapDumpOnOutOfMemoryError -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20

What I’ve been up to for over 3.5 years

March 23rd, 2011

I’m sitting at the Miami International airport waiting for my flight back to London and looking at the past 3.5 years. I had a chance to work on a fantastic project that finally went live, 100% live. 3.5 years ago I joined Monochrome as a contractor to work on a project for a company trading aircraft parts. When starting, late 2007, no one was expecting it will take so much time to complete. So what is this monster?

In short: a CRM, sales support, warehouse management and stock control system. In detail: a system to manage customers, suppliers, create purchase and sales orders, invoices and full stock control system, basically to run the whole company, 2 branches actually. It is built mainly with Adobe ColdFusion 9, Adobe Flex 3 and Microsoft SQL Server 2008 R2. Almost 60,000 lines of ColdFusion code and 25,000 lines of MXML+AS3. This application is used on a daily basis by about 70 people in 2 different locations. The data between these two locations is exchanged in near real-time manner, US and UK offices get better overview of the business.

When we started at Monochrome we had to deal with a totally different creature than what it is now. This new application (called ICS) was about to replace one mans job, a green screen system written in COBOL for over 20 years. The old technology imposed quite a few limitations – the client really wanted to keep developing new user interface on top of existing database (yes, I know…) This was achieved with Liant Relativity Data Server, it takes flat COBOL data files and exposes them via ODBC (Liant was bought by Microfocus who recently introduced COBOL for .NET4). As we kept developing we never really experienced any problems with performance. But ICS had to go to live user testing at some point. That’s where the problems started. We experienced serious performance issues and a lot of server crashes. It turned out that the reason was the database driver, it was slow and couldn’t handle joins so we ended up running lots of small queries that in ideal world could be replaced with a view or something. It took as over 2 years get to the stage when we were feature complete but with massive performance issues. Phase one was complete anyway, the client was happy, he he expected this sort of problems. Mind you, the company itself evolved a lot to suit to the new processes we were developing and implementing – this partially is the reason why the initial phase took so long.

This is where the things changed slightly (from now on “we” means me and the client only). Second phase, moving away from the legacy database and putting final touches would happend on site. There was quite a few reasons for this decision but this blog is not the right place to get into the details. I was offered a contract, things got really interesting.

First, we had to decide what the final hardware setup is going to be like. We had 2 old Dell 2950 32 bit servers already – we needed more. So we have got a couple of Dell R410 64 bit, 8TB storage, 24GB RAM machines – these are now serving as primary servers while 2950s are used as backups. Each location has its own set of servers. For virtualisation we use VMWare ESXi 4 on 64-bit machines and ESXi 3.5 on 32-bit machines. Each server is running 2 Windows instances, one for the web server, one for the database (Windows 2008 R2 on ESXi 4 and Windows 2008, R2 is 64 bit only, on ESXI 3.5). So 8 Windows instances in total. Each setup is exactly the same (apart from the amount of memory assigned to the VMs). Primary servers have 10GB RAM assigned for SQL Server 2008 R2 and 3GB for ColdFusion 9. US database serves as a backup for UK and UK database serves as a backup for US – we are using SQL Server log shipping without a mirror server instances. On top of that we have an offsite backup for full and transaction log database backups.

Software architecture of the core application is pretty straightforward, no real magic there. Flex application created with Flex 3 and Cairngorm 2 talking to ColdFusion 9 using remote objects. Oh, and we have an AIR wrapper for the application so we can use it inside the browser and on the desktop). But I mentioned before: “It is built mainly with Adobe ColdFusion 9, Adobe Flex 3″. We also use Java, C#, Ruby and node.js. Quick rundown:

  • Java: we use Jasper Reports for forms printing – stuff like order acknowledgements, invoices
    seriously, never thought that reporting software can be used this way, Jasper Reporting with iReport is a fantastic product I would happily pay the money for (if I was the project owner and I was paying the bills…)
    Jasper generates PDFs for us, these get printed using Qoppa jPDFPrintCLI product. I have to say – if you ever have to print PDF files I can recommend Qoppa for fantastic support.
  • C#: we have a scanning process integrated into the workflow, you see, when aircraft parts are coming in they often come with printed certificates. These certificates have to be scanned and stored forever. We are using Twain.NET desktop application (sitting in the tray) talking to the Flex app via sockets
  • node.js: our system required a realtime messaging and object locking mechanism. We have initially used Red5 for this but once we learned about node.js there was no point keeping Red5 running. Our realtime messaging server was running just fine for last 12 weeks last time when I checked. And it has only 200 lines of code.
  • Ruby: we use it for a lot of maintenance tasks

Of course every project of such a big scale needs source control. At the beginning of the development SVN was the choice but with time we moved to Git, many reasons behind this decision – but Git feels much better ;) We have an automated build server running Ubuntu with Apache Ant connecting to Github and building the application on demand.

And in the end there is an HTML application for mobile devices. It gives access to the most important details of customers, parts, batches, suppliers and so on.

This was an interesting journey. In some spare time I’ll try to describe most interesting parts in more details.

We are boarding, see you on the other side…

Long time no see

November 23rd, 2010

It has been a long time since I last blogged anything here. There was a lot going on over last year. There’s been this massive project I was working on, ColdFusion, Flex, .NET, Java, Microsoft SQL Server 2008, moving away from green screen application and even some real time data synchronisation between two different geo locations. It’s been an interesting, very exciting journey which I’m going to describe here very soon. Hope some of you will find it interesting and who knows, maybe there’s some knowledge to take away?

Yet another ViewStack control for Silverlight

December 15th, 2009

In one of my recent Silverlight projects I had to use a ViewStack control. There is few of these already available here and there but I thought it might be fun to build new one from scratch. As image speaks more than words here is the sample.

Install Microsoft Silverlight

p>In one of my recent Silverlight projects I had to use a ViewStack control. There is few of these already available here and there but I thought it might be fun to build new one from scratch. As image speaks more than words here is the sample.

[silverlight: http://gruchalski.com/wp-content/uploads/2009/12/ViewStackTestApp.xap]

And here is the source code.

Hope this helps anyone.

Flex Builder 3 Linux Alpha 5 available on labs.adobe.com

November 25th, 2009

I’m sure everyone who’s interested in Flex Builder for Linux already knows that by following Twitter and Adobe JIRA. So what’s exactly happened? Today, just 5 days before Flex Builder alpha 4 was to expire Adobe released an alpha 5 update on labs.adobe.com which is going to expire in 401 days.

I have the gut feeling that Adobe is working on Flex Builder 4 for Linux but is not keen on sharing the info yet. Specially that there is an unofficial build available from here.

Good decision Adobe.

More information on current release here and here (release notes).

Printing data on multiple pages with Silverlight 4

November 20th, 2009

This week at PDC09 Microsoft unveiled latest Silverlight release – first beta of version 4. One of the new features is printing support. There is a great video introduction to printing published by Tim Heuer. It show the basics, after watching it my first question was how to print the data on multiple pages. Here is how I do it. This application is a cut down version of the app published by Tim on silverlight.net.

In this sample we are going to print list of people and display sample header showing page number and some random title. Let’s jump in.

Let’s start with XAML:

<UserControl x:Class="PrintingTest.MainPage"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
    xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
    xmlns:data="clr-namespace:System.Windows.Controls;assembly=System.Windows.Controls.Data"
    mc:Ignorable="d"
    d:DesignHeight="300" d:DesignWidth="400">
    <Grid x:Name="LayoutRoot" Background="White">
        <data:DataGrid
            ItemsSource="{Binding}"
            Height="247" HorizontalAlignment="Left"
            Margin="12,41,0,0" Name="dataGrid1"
            VerticalAlignment="Top" Width="376" />
        <Button
            Content="Print data" Height="23"
            HorizontalAlignment="Left" Margin="12,12,0,0"
            Name="PrintDataBtn" VerticalAlignment="Top"
            Width="126" Click="PrintDataBtn_Click" />
    </Grid>
</UserControl>

Next is the Person class which is exactly the same as in the original video:

using System;

namespace PrintingTest
{
    public class Person
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public int Age { get; set; }
        public string FullName {
            get {
                return string.Concat(FirstName, " ", LastName);
            }
        }
    }
}

And now the most important part – the code behind. The MainPage class wires up the Loaded event and when the app is loaded it populates the list of people as well as sets the DataContext on LayoutRoot.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Printing;
using System.Windows.Media;

namespace PrintingTest
{
    public partial class MainPage : UserControl
    {

        public MainPage()
        {
            InitializeComponent();
            Loaded += new RoutedEventHandler(MainPage_Loaded);
        }

        void MainPage_Loaded(object sender, RoutedEventArgs e)
        {
            List<Person> people = new List<Person>();
            for (int i = 0; i < 20; i++)
            {
                people.Add(new Person() { FirstName = "Al", LastName = "Pacino", Age = 69 });
                people.Add(new Person() { FirstName = "Robert", LastName = "De Niro", Age = 66 });
                people.Add(new Person() { FirstName = "Val", LastName = "Kilmer", Age = 50 });
                people.Add(new Person() { FirstName = "John", LastName = "Voight", Age = 71 });
                people.Add(new Person() { FirstName = "Tom", LastName = "Sizemore", Age = 48 });
            }
            LayoutRoot.DataContext = people;
        }

        private void PrintDataBtn_Click(object sender, RoutedEventArgs e)
        {
            // header height:
            double headerHeight = 70.0;
            // footer height:
            double footerHeight = 30.0;
            // single line height:
            double tboxHeight = 20.0;
            // how many elements can we place on one page?
            int numElements = -1;
            // current page number:
            int page = 1;
            // total number of pages to print:
            int totalPages = 0;
            // number of the current item:
            int counter = 1;

            PrintDocument doc = new PrintDocument();
            doc.PrintPage += (s, args) =>
            {
            };
           doc.Print();
        }

    }
}

The code below is responsible for printing and should be placed inside doc.PrintPage lambda. Let’s discuss what is going on there first.

When the application attempts to compose first page the number of elements per page and total number of pages is calculated. Next, the header is created. It is a Grid with two TextBlocks. First of the text blocks holds the title while the second one displays current and the total pages numbers; header is created separately for each page.

Once the header is constructed the dynamicPanel StackPanel is created and previously constructed headerGrid is added to its Children list. Next, we query dataGrid1.ItemsSource for the first of people to display on current page. We skip as many elements as we fitted on previous pages and take number of elements for the current one.

The penultimate step is enumerating over ppl list and for each one of them create a StackPanel with two TextBlocks. First displays the name of the person, second one the age. Finally we set HasMorePages.

HasMorePages is the key to multiple pages printing. As long as it is set to true Silverlight expects more pages to be printed. That’s why when the number of current page is equal to total number of pages we set it to false. At this point printing the document is finished.

Here is the code:

                if (page == 1)
                {
                    numElements = (int)((args.PrintableArea.Height - headerHeight - footerHeight) / tboxHeight);
                    double totalItems = (double)((List<Person>)dataGrid1.ItemsSource).Count;
                    totalPages = (int)Math.Ceiling(totalItems / (double)numElements);
                }

                Grid headerGrid = new Grid()
                    {
                        Width = args.PrintableArea.Width,
                        Height = headerHeight
                    };
                headerGrid.RowDefinitions.Add(new RowDefinition());
                headerGrid.ColumnDefinitions.Add(new ColumnDefinition());
                headerGrid.ColumnDefinitions.Add(new ColumnDefinition());

                TextBlock headerTitle = new TextBlock()
                {
                    Text = "Contact list",
                    HorizontalAlignment = System.Windows.HorizontalAlignment.Left,
                    VerticalAlignment = System.Windows.VerticalAlignment.Center,
                    Margin = new Thickness(50,0,0,0),
                    FontSize = 20
                };
                TextBlock pageCounter = new TextBlock()
                {
                    Text = string.Concat("Page ", page, " of ", totalPages),
                    HorizontalAlignment = System.Windows.HorizontalAlignment.Right,
                    VerticalAlignment = System.Windows.VerticalAlignment.Center,
                    Margin = new Thickness(0, 0, 50, 0)
                };
                headerGrid.Children.Add(headerTitle);
                headerGrid.Children.Add(pageCounter);
                Grid.SetColumn(headerTitle, 1);
                Grid.SetColumn(pageCounter, 2);

                StackPanel dynamicPanel = new StackPanel();
                // add header to display list:
                dynamicPanel.Children.Add(headerGrid);

                IEnumerable<Person> ppl = ((List<Person>)dataGrid1.ItemsSource).Skip((page - 1) * numElements).Take(numElements);

                foreach (Person p in ppl)
                {
                    // create stack panel for each person:
                    StackPanel namePanel = new StackPanel();
                    namePanel.Orientation = Orientation.Horizontal;

                    TextBlock counterBlock = new TextBlock();
                    counterBlock.TextAlignment = TextAlignment.Right;
                    counterBlock.Margin = new Thickness(0, 0, 5, 0);
                    counterBlock.Text = string.Concat(counter, ".");
                    counterBlock.Height = tboxHeight;
                    counterBlock.Width = 50.0;

                    TextBlock nameBlock = new TextBlock();
                    nameBlock.Text = p.FullName;
                    nameBlock.Height = tboxHeight;

                    TextBlock ageBlock = new TextBlock();
                    ageBlock.Text = string.Concat("who is ", p.Age, " years old");
                    ageBlock.Foreground = new SolidColorBrush(Colors.Red);
                    ageBlock.Margin = new Thickness(5, 0, 0 , 0);
                    ageBlock.Height = tboxHeight;

                    namePanel.Children.Add(counterBlock);
                    namePanel.Children.Add(nameBlock);
                    namePanel.Children.Add(ageBlock);
                    dynamicPanel.Children.Add(namePanel);

                    counter++;
                }

                args.HasMorePages = !(page == totalPages);
                args.PageVisual = dynamicPanel;

                page++;

Full source code can be downloaded from here. You can test working example below, please make sure you use Silverlight 4 beta. It can be installed from this location.

Read more »

Installing and connecting Apple Built-in iSight in Windows 7 x64 running under VMWare Fusion

November 19th, 2009

It took me a while to figure out how to install Apple Built-in iSight driver so it could be used in Windows 7 x64 running inside VMWare Fusion. I have finally found a solution here.

Installing the drivers:

  1. run cmd.exe as Administrator
  2. D: (or whatever the drive letter of mounted CD is)
  3. cd "Boot Camp\Drivers\Apple"
  4. msiexec /i BootCamp64.msi

It will take a while so be patient. Once installed, restart Windows 7 and you are ready to go.

To connect Apple Built-in iSight:

  1. VMWare Fusion > Virtual Machine
  2. USB
  3. Connect Apple Built-in iSight

Don’t forget to enable USB sharing.