Home > Development Projects

Introduction

This page contains links to programs developed by me. Currently I am maintaining two web portals, the first one being here, the second one at MSDN Code Gallery. The purposes of them are:

This Site: Contains miscellaneous programs. Personally I think a blog will be a good place to put them, but it requires a respectful amount of time to maintain.

Powershell Tweaks (at Code Gallery): This portal maintains various topics on customizing Windows Powershell. Windows Powershell is a object-oriented command line terminal introduced by Microsoft in late 2006. It mimics Bash shell syntactically, but has an amazing .NET based scripting facility. For Windows workstation users, it is a necessary tool to empower their work environment. The goal of tweaks is to augment the functionality of Powershell so that it feels more like a Unix environment.

Tools

My working environment is a laptop with Windows Vista, although I do work with Unix, GNU/Linux and FreeBSD through SSH or VMware. Despite the various kinds of platforms I use, I like to work in a single system whenever possible. This leads to my adoptation of Windows Powershell, whose scripting capability is strong enough to replace bash under GNU/Linux. In fact, one of my driving motivation on development is to provide equivalent utilities that can be used in Windows, such as sudo. I always visit sourceforge and codeproject and try to examine the code in various packages.

Licence

The programs on this website, if not specially specified, are released under GNU General Public Lincense V3.

If you have any problem, feel free to contact me.

This Site

* Powershell Build Environment (December 27, 2009)

I have been using Windows Powershell as my working shell for some time. It offers a UNIX-like command line capability with an object-oriented scripting language. There is nothing it cannot do whereas cmd.exe can.

Unfortunately, it seems that cmd.exe is still and will continue to exist with Powershell for several years to come. Most development tools such as Visual Studio, Intel Compiler, DirectX SDK and Windows Driver Kit install their own version of command line build environment based on cmd.exe. These tools have their own environment variables configured by bat or cmd scripts. It would be a good thing if there is a single script that plugs the build environment variables into powershell anytime you want.

Here it is. Download this script (dev.ps1) and put it in your PATH environment variable, preferably somewhere dedicated for scripts, such as ~/scripts. Then modify its content by adding bat stubs that call the real bat or cmd scripts. Basically, all build environment are characterized by their environment variables, and the only thing we need to do is to run the bat or cmd scripts (such as vcvarsall.bat for Visual Studio) and dump the variables, and lastly reinstall them into the powershell environment. Change the variable $_tocall such that it points to where bat file is located.

Execute dev.ps1 and pass arguments as you wish.

Download dev.tar.gz

* Unicode Support for Offline Wikipedia (August 20, 2009)

Wikipedia is one of the most comprehensive free references at hand. Without having to take a dip in the library for hours looking up a definition, one could get the ever-updating information on whatever he/she wonders.

Unfortunately, it requires you to be online and a little trouble is there if you do not have a fast internet connection.

If you have a few tens of gigabytes free on your hard drive, and if you have a GNU/Linux installation, be it bootable or installed as a virtual machine, here is a good article about how to make your own offline wikipedia.

I followed it, and successfully got the English and Chinese version to work after a quarter of day's extraction and indexing. However, it fails miserably when the search keyword or the link contains a non-ASCII character. The error is:

UnicodeEncodeError at /article/操作系统/ ('ascii', u'./quickstartsearch db/ "\u64cd\u4f5c\u7cfb\u7edf"', 24, 28, 'ordinal not in range(128)')

Fortunately, this seems to be a Python issue, which I solved after investigate. Nevertheless, it is again another misery since I am totally ignorant of Python.

For anybody who do not want to read the following lengthy description of how I got it to work, please download and replace the views.py file. It contains comments that describes what has been changed.

The problem is caused by the failure to encode the non-ASCII characters for use in os.popen function, in the function definition of article(request, article) in views.py. The function invokes a shell command, piping the output to stdin. It does not support commandline strings with non-ASCII characters. To make matters worse, it became obsolete long ago and there is defintely no fix in further versions.

Its successor, the subprocess.Popen function, behaves in a different way. First, it accepts non-ASCII characters. Second, one of its arguments, shell, must be assigned True (which defaults to False) when the commandline string is to be invoked. Third, the output can be piped to places other than stdin, so stdout=PIPE must be used. The returned object, Popen, contains a function called communicate(), which returns [stdout, stderr]. We make use of the first attribute.

OK. The next step is to make sure that the replacement works smoothly. Unfortunately, due to the weird behavior of subprocess.Popen from its prodecessor, we cannot leave other part in the code file. Popen.communicate()[0] behaves like a pipe, so a for loop will pick one byte a time, not one line a time as we wish. Instead, we obtain the pipe output as a whole string, splitting it with the newline character, and appending a newline character to each item to ensure the coherence (i.e. type and format) of the line object. Next, we want to make sure that the regex match works well. This proves to be true. However, since the returned type is str (byte array), another conversion will fail in the next comparison, res.group(3) == article.tolower().replace('_', ' '), in which the latter is unicode string. We encode the unicode string to UTF-8 to let it work.

Donwload views.py

* Update: new sudo for Windows Vista and above (July 21, 2009)

Last time I mentioned that there are two ways to create an elevated process in Windows Vista. The ShellExecute approach does not require that the quasi-parent process be elevated, since the actual parent process is the explorer. The downside is that a new Window must be created. By contast, the system service approach can overcome this shortage, but it can expose security vulnerabilities if improper authentication is applied.

Someone in CodeProject has come up with a detailed analysis on the User Access Control feature of Windows Vista, and published a series of APIs that can be used for a Unix-like sudo utility on Windows. The idea is still based on ShellExecute function, but it hides the new Window and pipes I/O to the parent process. This hack is awesome, and I have updated my sudo program to a more Unix-like one based on it.

The package consists of three binaries. sudo.exe is the main executable, with Elevate.dll and Elevate.exe the support files. They have to be placed under the same directory to work.

The support binaries are provided with no sources. They can be found on the author's CodeProject page.

To have a try, open an unelevated console (either cmd or powershell), type

.\sudo fsutil

compare the output with

fsutil

Download source code with binaries

* uptime for Windows (July 12, 2009)

In Unix systems, uptime utility gives the accumulative time since last boot. This can also be achieved on Windows Platforms before Windows Vista. In Windows XP, for example, one types

systeminfo | find "Up Time"

to see the uptime.

Since Vista, however, this information is no longer available in systeminfo. Rather, it is provided in Task Manager under the Performance Tab. Since I am not pleased with such a change, uptime for Windows is born. Simply type " uptime" and see the result.

Download source code with binary

* sudo for Windows Vista (July 5, 2009)

Windows Vista features a brand new facility called User Access Control, as a response to critiques on the security of its predecessors. The main idea is the least privilege principle, which means a process should be granted no more privilege than enough to maintain its normal operation.

As a result of such security attributes, a process launched by a double click in the (non-elevated) shell runs under a restricted privilege, even if the user has administrator privilege. To alter such a behavior, the user has to either run an elevated shell (by launching an elevated Task Manager, killing the process explorer.exe in it, and launching a new process explorer.exe), or right-click the item to be launched and select "Run as Administrator".

At first glance, it does not take more steps than those needed to launch a normal process. However, for console (powershell or cmd) applications, this means that certain utilities, such as fsutil and Create-Link do not work unless the console process is elevated.

Having read through the documentation on CreateProcess and CreateProcessAsUser, some conclusions can be drawn. The most important among them is that a new process has no more privilege than its parent process. This simply puts off the idea of a Unix-like sudo, since it must be a child process of the console.

There are two alternatives. One of them is to create the new process under the session user account if the user is an administrator (like sudoers in Unix). This solution comprises a service process and a front-end process. The front-end process is created by the console process, invoking the service process, which authenticates the user and creates the elevated process. The I/O of the elevated process is to be associated with the console shell. A good implementation of  this idea is the sudowin application.

A much simpler workaround, if one is willing to give up the current console and launch a new one, with the help of ShellExecute function. With a special attribute set up, ShellExecute creates a new elevated process under the shell process. This is the same approach as that of right-click and select "Run as Administrator".

Here is an implementation of the second idea. sudo searches an executable (ending with .exe) in the current working directory and the PATH environment variable, and launches an elevated process.

Download source code and binary

* Update: passing wide character strings as arguments directly (June 21, 2009)>

Last time I discussed two alternatives on passing wide character as arguments that did not work, and one way to the solution was to use a converter function. This time I upload a piece of code that do work, using TCHAR.h routines. It is specific to Microsoft Visual C++, and therefore all routines within it starts with an underscore `_'.

Download the code that works

* uname for Windows 2000 and later (Jun 20, 2009)

uname is a program under Unix and Unix-like systems. It is used to display information on the operating system name, kernel, revision, hardware platform and processor information. A more detailed explanation can be found here.

The uname utility that comes with Gnuwin32 is less than trivial: it simply prints the system name and is not as rich as it does on Unixy systems. Therefore, I wrote a new version of uname based on a code example at MSDN that checks the OS type. It fully complies to the POSIX standard on uname.

The usage is (also available by typing `uname --help'):

Usage: uname [OPTION]...
Print certain system information. With no OPTION, same as -s.

    -a, --all                                  print all information, in the following order,
                                               except omit -p and -i if unknown:
    -s, --kernel-name                          print the kernel name
    -n, --nodename                             print the network node hostname
    -r, --kernel-release                       print the kernel release
    -v, --kernel-version                       print the kernel version
    -m, --machine                              print the machine hardware name
    -p, --processor                            print the processor type or "unknown"
    -i, --hardware-platform                    print the hardware platform or "unknown"
    -o, --operating-system                     print the operating system
        --help                                 display this help and exit
        --version                              output version information and exit

Report bugs to George.

It runs on Windows 2000 and up to Windows 7 (and Windows Server 2008 R2).

Download binary with source

* Create-Link

This program creates a hardlink or symbolic link on a drive. The definitions for hard link and symbolic link can be found on Wikipedia.

By default, Windows Vista has a mklink command which performs the same task, but only available in cmd. If someone is seeking for an equivalent tool that can be used with Windows Powershell, that's it. The name takes the form of a cmdlet, although the program is written in a mixture of C and C++. It is also possible to write a real cmdlet, by using C#'s ability to interop with native C code. Writing in C requires more effort, most notably, the problem with converting char* to LPCWSTR. I have encountered this problem and tried to search a solution, but couldn't find the piece of code that finishes that task. Therefore, after I solved the problem, I decide to publish it below.

This program runs on Windows Vista and above. It requires an administrator session (run a Powershell or cmd as Administrator).

Donwload binary with source

* How to convert char* to LPCWSTR?

Anyone who writes Visual C++ with Windows API has to deal with the conversion between a zero-terminated char* string and a LPCWSTR. This is because the main function is main(int argc, char** argv), in which argv is an array of char string of multi-byte characters. However, Windows (NT) API uses almost exclusively wide character functions, which means that the string arguments are represented by an array of wide characters. I don't explain the difference between them, since a lot of people have already did it, and if you don't find them, this may help (see the first two chapters). What I want to do is to give a function that does this job.

View code (May 9, 2009)

Update (May 15, 2009): I came into another workaround somewhere, though haven't tried yet. Declare args as TCHAR** type, which aligns to the chararacter type used by the windows header file. In other words, if UNICODE is defined, TCHAR indirects to wchar_t, otherwise char. However, I tried to declare args as LPCWSTR* before, but it doesn't work.

Update (May 19, 2009): The workaround I mentioned above DOES NOT work. I used the following code and got garbled output.

View code that doesn't work (May 19, 2009)