Category Archives: Random

The random posts other than an essay, can be anything, a quote for example.

Welcome Lancelot

Some words

It’s been very busy these days, I was very busy doing my projects, and use my spare time create a proxy tool for Cocoa, called iProxy.

iProxy is a smart SSH tunnel proxy that can using SSH tunnel to bypass the firewall and using the PAC to choose which url should go through the proxy, and it can also choose free http proxy too.

I haven’t release it yet, since I’m coding for the smart PAC part and try to figure out how to charge for that, of course, the iProxy client is open source too.

That’s the reason that I haven’t update this blog for a very long time(about 2 months), after the launch of iProxy, I maybe have time to sit down and write something about the Objective-C and cocoa programming, and for now, I just need to perfect iProxy and make it more like a product, not a toy.

This is the additional words for this blog, this blog is going to announce the scripting tool I told before at my last blog.

This script tool, I called it lancelot. The details, is as below. You can find the code of lancelot at here: https://github.com/guitarpoet/lancelot

Wish you will like it, and try for it, at least, I’m enjoying using it in my projects now.

Lacenlot, tool for manage multiple daemon processes

Introduction & Purpose

The purpose for lancelot is to help to launch and mange the worker processes for
my working project.

Lancelot is used to solve the problem I encountered in my project:

  1. I need to launch many worker processes as daemon process on many slave \
    machines (say, 5 worker for each slave machine)
  2. After some configuration change, I need to restart all the worker processes \
    (sure, you can use apache zookeeper to handle this, but it will make the \
    worker process more complex, since the worker process just need to do some \
    very simple work (get the data from an url and store it into hdfs for example)
  3. I need to check for all the worker process’s status at a period of time \
    (or maybe I want to get an alert when 50% of the processes is down)

So, lancelot is used for:

  1. Launching as many worker processes as you want, and save the pid files as the \
    template you have gave it (something like /var/run/worker-1.pid), and the \
    redirect the standard output, error output to the templates you have gave it \
    (somthing like /var/log/worker-1.log /var/log/worker-1.err)
  2. Launching the worker processes as daemon using nohup or daemonize, lancelot \
    prefer daemonize than nohup, anyway, the worker process won’t exit until it \
    gets an error or itself should exit
  3. Check the status of all the worker process you have launched, using a ps \
    format, you can also set the ps format, you can use grep pattern or pid \
    template to check for the status
  4. Kill all the processes you want to kill by using a pid template or the pattern

And, for the ease to deploy lancelot to multiple slave machines, lancelot also has
a deploy function to deploy it to all the machines, this function is depend on
another script tool called taktuk.

Installation

For the installation, just unpack the package and copy lancelot to the destination
you want to put it to(/usr/local/lancelot for default).

And change the lancelot home variable in the lancelot script. Then add or ln lancelot
script to the PATH (usually /usr/local/bin/lancelot)

And you can use it.

Usage

You can use lancelot script like this

Lancelot command options

For example:
lancelot launch -c ~/launch_config -t 10 /bin/sh fetch_data.sh

Lancelot Launch

Description:     
    The process launching script.    
    Example:    launch -o /tmp/a-{}.log -e /tmp/a-{}.err -w /tmp /bin/ls .
    Email to guitarpoet@gmail.com if bug found.
Options:     
    -w|--workingdir:        The working dir for the launch
        This option requires 1 args.
    -o|--output:
        The standard output of the application
        This option requires 1 args.
    -e|--error:
        The error output of the appliaction
        This option requires 1 args.
    -p|--pid:
        The pid template for the application
        This option requires 1 args.
        Option is required.
    -c|--config:
        The configuration file location for lancelot launch
        This option requires 1 args.
    -t|--count:
        The process count that want lancelot to launch
        This option requires 1 args.

Lancelot Status

Description: 
    The process status script.
    Example:
    status -p /tmp/a-{}.pid
    Email to guitarpoet@gmail.com if bugs found.
Options: 
    -p|--pid:
        The pid template for the application
        This option requires 1 args.
        Option is required.
    -a|--application:
        The application pattern to grep
        This option requires 1 args.
    -c|--config:
        The configuration file location for lancelot launch
        This option requires 1 args.
    -f|--format:
        The ps format for this application, 'pid args' is by default
        This option requires 1 args.

Lancelot Kill

Description: 
    The process kill script.
    Example:
    kill -p /tmp/a-{}.pid
    Email to guitarpoet@gmail.com if bugs found.
Options: 
    -p|--pid:
        The pid template for the application
        This option requires 1 args.
        Option is required.
    -a|--application:
        The application pattern to grep
        This option requires 1 args.
    -c|--config:
        The configuration file location for lancelot launch
        This option requires 1 args.

Lancelot Broadcast

Description: 
    Broadcast exec the lancelot command to all the hosts
    Example:
    broadcast -h ~/hosts status -p /tmp/a-{}.pid
    Email to guitarpoet@gmail.com if bugs found.
Options: 
    -c|--config:
        The configuration file location for lancelot broadcast
        This option requires 1 args.
    -h|--hosts:
        The hosts that should launch the lancelot broadcast
        This option requires 1 args.
    -l|--login:
        The login name for login using taktuk
        This option requires 1 args.

The launch of Jersey

I was glad to announced that the launch of my JavaScript execution environment(called Jersey, in the name of JavaScript of Easy).

This begin with the thought on how to get a handy too to handle the tiny programming works I have to do very often, and has many foundation between(such as text manipulating, http site data fetch and processing, xml processing, html processing, sold data updating or data migration).

I’ll write an article to explain the this later.

The link for Jersey is https://github.com/guitarpoet/jersey and the README of it is below.

Jersey: A pluggable JavaScript execution environment

This is a pluggable JavaScript execution environment based on Rhino. It can run on any Java runtime above Java 5.

Jersey also contains a package manager called jpm, so you just need to provide an ivy module file, then you can get all the plugin and plugin dependencies using maven distribution.

Concept

Jersey is a running environment for JavaScript, it can run Mozilla Rhino flavoured JavaScript using Java. The aim for Jersey, is to provide a nice and pluggable environment to playwith or use JavaScript and based on the richi set of Java libraries.

The concept of Jersey is as list bellow:

  • Pluggable: Jersey is pluggable, you can install any plugin to it, and the plugin is just a java jar. All the plugins are deactivate at the start of the enviromnent, you can activate the plugins using Spring(Jersey provides you the functions to do that).
  • Simple: Jersey try to provide the api as simple as possible, and provide a console to play JavaScript with and will add the support of other script that compiles to JavaScript(for example, CoffeeScript) using the console, and the console support the tab completion, so you can fool around very easily
  • Configurable: Jersey provides the configuration basics based on Java’s properties, and all the configuration for application and plugins contains within the same configuration file
  • Discoverable: You can list all the provided function and provided modules using native functions.
  • Docable: Jersey provides the doc method for all the function and plugin modules, you can read the documentation of the function and modules using man function, and even have a pager function to view the pages.
  • Easy to be server: Since Java has many good embedded services, Jersey using Apache Derby to provide Database service and using Jetty to provide http server service, I also have a small CMS system written using the mvc and http plugin of Jersey
  • Easy to use: Jersey providing the pacakge management system based on the Famous Maven and Ivy, so you can download every plugin and plugin dependencies using Ivy’s module file.
  • Can support other languages: Since JavaScript is not a very simple language to write, Jersey also supports Coffee script(detected by the file extention) and using coffee to compile it and run, will support others in the future
  • Powerful: Jersey is based on Java, so it can take advantage of current Java and opensource Java libraries to provide powerful functions, it can use JDBC to access Database, using Commons DBCP to make the database connection pool, it can use Solr client to access the Apache Solr, it can use Log4J to handle the logging things and even using JBoss Drools to do the Rule matching.
  • Stable: Thanks to Java’s stablility and JavaScript’s simplicity, Jersey can be very stable for running, it can run very long time without memory leaks or make system unstable. Jersey’s multithreading is using Java’s multithreading support, and Jersey provides a JobManager function using Java’s concurrent library.

Architecture

Jersey is based on Java, Rhino and Spring, the component structure of Jersey is based on Spring, and the interpreter of JavaScript is using Rhino.

All the native functions and native modules is initiliazed at the intializing of the environment, all the plugins you want to use, need to be installed into the lib folder first, configure it correctly in the configuration file, then using require function to require the intialize script for this plugin.

Usage

Jersey is used using commandline, you can run the console using just jersey command. Or jersey a.js b.js c.js to run the script file one by one.

Jersey accept -c option to get the option file location (default is config.properties), and -s to get the script file(This only used for standard in, standard in file for Jersey is – )

Native Functions

Here is docs for some native functions that Jersey provided, you can list all the native functions using function functions().

  • require: Require the library using the resouce location, support file:file and classpath: protocol.
  • appContext: The function to load application context to javascript console
  • config: Read the configuration from system.properties or list all the properties
  • functions: List all the functions this shell provided.
  • modules:Prints all the modules that have loaded.

Native Modules

Here is the docs for some native modules that Jersey provided, you can lis all the native modules using function modules().

  • console: The console object.
  • file: The file utils.
  • csv: The csv utils service.
  • sutils: The string utils service.

Standard Libraries

Jersey provides some standard JavaScript libraries in the distribution. Here is the list of the libraries:

  • std/common: This provides the common extention of JavaScript, for example capitalize, isBlank, endWith for JavaScript and so on.
  • std/date: This provides date
  • std/evn.rhino: This provide env, to use library like jQuery
  • std/man: This provide the manual function for all the native function and modules
  • std/jquery: This provides the famous jQuery
  • std/underscore: This provides underscore

You can load your scripts using classpath protocol too.

Funny Quotes About Programming

I was looking for quotes for the second part of thinking about JavaScript, and found many interesting quotes about programming at here.

Here is some that I loved, they shows how intelligent and humour the author is.

 “A C program is like a fast dance on a newly waxed dance floor by people carrying razors.” – Waldi Ravens.

C is not really friendly to the beginners of the programming(at least to me), just like guns to the kids, they will hurt themselves eventually.

“I think Microsoft named .Net so it wouldn’t show up in a Unix directory listing.” – Oktal

To Unix guys(even for Java guys), .Net seems to be a hell. C# is a good language(no wonder), it has a mix nature of many languages(including Java), it has many good natures but it provides too many ways, so there are so many questions on StackOverFlow, just to ask how to use the some of the syntax of C# Correctly!!!. And for libraries of .Net, I have to admit, that’s a nightmare. The OpenFileDialog relative path bug, it exists until this day (yes, this is a tiny bug in all the misdesign and bugs of .Net framework).

“Fine, Java MIGHT be a good example of what a programming language should be like. But Java applications are good examples of what applications SHOULDN’T be like.” – pixadel

This is true before SWT, no one can say Eclipse is a bad example for applications. Even Swing do much better today