Archive for the ‘Programming’ Category

JScript to automatically retrieve / check web page

Tuesday, January 12th, 2010

Recently I discovered a problem with Manifold WMS where if the page wasn’t accessed for a while it would cause the w3wp.exe IIS worker thread to loop. Until I find out the underlying cause I’ve written the map_keepalive.wsf script below to automatically retrieve a web page and scheduled it to run every 5 minutes. I assume something gets spun down and the concurrent WMS requests cause a problem, the map_test.asp page that this script retrieves is just a default IMS page generated from within Manifold.

<package>
  <job id="js">
     <script language="JScript">
	var finished = false;
	var timeout = 0;
	var timeoutSecs = 30;
	var forAppending = 8;
	var xmlhttp = WScript.CreateObject("MSXML2.XMLHTTP");
	xmlhttp.open("GET", "http://www.satsleuth.com", false);
	xmlhttp.onreadystatechange = function() {
		if(xmlhttp.readyState == 4)
			finished = true;
	}
	try {
		xmlhttp.Send("");
	} catch (e) {
	}
	while ((!finished) && (timeout < timeoutSecs))
	{
		timeout++;
		WScript.Sleep(1000);
	}
	var msg = ""
	if (!finished)
		msg = "Timeout occurred";
	else if (xmlhttp.status != 200)
		msg = "Error retrieving map_test.asp, error code: " + xmlhttp.status;
	if (msg != "") {
		var oFSO = WScript.CreateObject("Scripting.FileSystemObject");
		var oFile = oFSO.OpenTextFile("map_keepalive.log", forAppending, true);
		var dt = new Date();
		oFile.WriteLine(dt.toLocaleString() + ": " + msg);
		oFile.Close();
	}
      </script>
   </job>
</package>

Manifold GIS IMS (Internet Map Server) error 0x80004005

Wednesday, December 2nd, 2009

I’m working on an iPhone 3GS mapping application using Manifold as a map tile server via the Web Map Service (WMS) protocol. I did initial development using a Vista x64 development machine on my intranet via wifi but had considerable difficulty moving to my Windows Server 2008 64-bit machine so that I could access the map tiles remotely via the 3G network. It’s been a while since I’d deployed a Manifold IMS solution but I remembered the usual steps:

  • Web site must be running under 32-bit mode because the COM object is 32-bit only
  • Map files, config.txt and any linked resources must be available to the ‘NETWORK SERVICE’ account
  • Same account needs access to Windows\serviceprofiles\networkservice\AppData\Local\Temp

    After my initial attempts failed I did numerous searches of the Manifold site and other resources and worked through the Manifold “Problems with the Internet Map Server” guide but kept getting the 80004005 error that I thought pointed at a permissions error. I thought to install the excellent SysInternals Process Monitor utility and the first error that became apparent was a file permissions error writing to the file “c:\Manifold.log”. I created a blank file with that name and granted full control permissions to the NETWORK SERVICE account used by IIS 7 and found several errors including the following:

    Invalid component name
    Incompatible component type
    Can’t generate unique name for temporary file

    Creating a new blank map file solved the problem so it appeared to be a problem within the Manifold map file itself, I decided to copy back a fresh copy of the map from my development machine so I could try removing a few elements until it worked but found it worked first go with the fresh copy. I didn’t modify anything with the map file so can only conclude it got corrupted at some initial stage while I had configuration problems. Anyway to avoid similar problems in the future I’ll make sure that during Manifold IMS deployment I always include a blank “c:\Manifold.log” file with appropriate permissions and be careful to archive any changes no matter how small they seem in source code control as a recovery point.

    I love Manifold GIS in general and it represents great value for money, but it would be nice if the object model was structured so that it would throw back a response code in the form of a text message on the call to MapServer.Create so it could be passed back to the client browser. Additionally rather than the non-standard practice of writing a text file to the root directory it would also be nice to see it write a meaningful errors to the Windows application event log.

  • Using JET OLE DB under ASP and Vista x64

    Sunday, November 1st, 2009

    I’m currently performing some work on classic ASP pages that during development are retrieving data from an existing Microsoft Access database. After receiving the following error I found that Microsoft haven’t and have no plans to release JET OLE DB drivers for 64-bit platforms. Fair enough it’s hardly the way of the future but I wanted to be able to perform some development on the pages before migrating to SQL/Server.

    Provider=Microsoft.Jet.OLEDB.4.0
    ADODB.Connection error ‘800a0e7a’
    Provider cannot be found. It may not be properly installed.

    After some digging around I found the solution was to change the IIS 7 settings to run the application in 32-bit mode. That can be performed under Vista x64 using the following steps:

  • Click Start > All programs > Administrative tools > Internet Information Server (IIS) Manager
  • Under the machine name click on Application Pools.
  • Right-mouse click on DefaultAppPool and select Advanced Settings…
  • Under the general section set “Enable 32-Bit Applications” to true.
  • Press OK and IIS should now be running in 32-bit mode, no other actions are necessary.
  • Getting detailed ASP errors under IIS 7.0

    Thursday, October 29th, 2009

    Some time ago I was doing maintenance work on some classic ASP pages and couldn’t get detailed error reports sent back to the browser. Using Internet Explorer I disabled “Show friendly HTTP error messages” but still received the generic HTTP 500 type error messages from IIS 7.0 running under Windows Server 2008. The same also occurred under FireFox. The modifications I was making to the ASP pages were fairly limited so I gave up on trying to get it to work and ending up sticking with examining the IIS logs on the server for debugging information.

    Today I started work on maintaining some more complex ASP pages developed by someone else under Vista x64 and IIS 7 and encountered the same problem. I also noticed Vista locks the IIS log files while the web server is running which further added to my determination to get ASP errors including line numbers back to the browser. It appears setting “Send Errors To Browser” in “Debugging Properties” for the ASP properties doesn’t work at a virtual directory level, it only works when enabled at the root level for the entire site. Enabling it at the higher level got back the detailed error reports as per the default bahaviour of IIS 6.

    ASP site map generator

    Tuesday, October 27th, 2009

    This is a simple ASP site map generator that I wrote some time ago using VBScript, it’s for a human-readable sitemap not the XML sitemaps used by Google and other search engines. It uses the Scripting.FileSystemObject to scan the same directory level that the ASP is placed at and returns a link to any htm or html pages found along with the last modification time and the title of the page. It was fairly quickly written so the parsing used for the title tag is just to suit the format I was using and it might need some other tweaking to suit your own site.

    I’ve just loaded a copy at http://www.satsleuth.com/site_map.asp so you can take a look at the script in action. Because I don’t actively use the script anymore I haven’t updated to XHTML 1.0 Strict so on the W3C Markup Validation Service it will only pass as HTML 4.01 Transitional. Currently presentation elements are embedded in the HTML rather than a CSS style sheet. Here is the main logic for the page:

    <table border="1">
    <tr><td><b>Page name</b></td><td ALIGN="RIGHT"><b>Modified</b></td><td><b>Title</b></td></tr>
    <% 
    	Dim objFSO, objFolder, objFiles, objFile, strFile, Title, CurLine, LCaseLine, TitlePos, PageCount, FontColour
    
    	Private Sub GetTitle(FileName)
    		Dim fso, file, strPhysicalPath
    
    		Title = ""
    		set fso = Server.Createobject("Scripting.FileSystemObject") 
    		strPhysicalPath = Server.MapPath(FileName) 
    		set file = fso.opentextfile(strPhysicalPath, 1)
    		do until file.AtEndOfStream or Len(Title) > 0
    			CurLine = file.ReadLine
    			LCaseLine = LCase(CurLine)
    			TitlePos = InStr(LCaseLine, "<title>")
    			if TitlePos then
    				Title = Right(CurLine, Len(CurLine) - TitlePos - 6)
    				Title = Left(Title, InStr(LCaseLine, "</title>") - 8 )
    			end if
    		loop
    		file.close
    	End Sub
    
    	Set objFSO = Server.CreateObject("Scripting.FileSystemObject") 
    	Set objFolder = objFSO.GetFolder(Server.MapPath("."))
    	Set objFiles = objFolder.Files
    	PageCount = 0
    	For Each objFile in objFiles 
    		strFile = LCase(objFile.Name) 
    		If Right(strFile, 4) = ".htm" or Right(strFile, 5) = ".html"  then
    			GetTitle(objFile.Name)
    			if Len(Title) > 0 then
    				PageCount = PageCount + 1
    				if objFile.DateLastModified >= Date - 7 then
    					FontColour = "<font color=""#FF0000"">"
    				else
    					FontColour = "<font color=""#000000"">"
    				end if
    				Response.Write("<tr><td><a href=""" & strFile & """>" & _
    					strFile & "</a></td><td align=""RIGHT"">" & _
    					FontColour & DateValue(objFile.DateLastModified) & "</font>" & _
    					"</td><td>" & Title & "</td></tr>")
    			end if
    		end if
    	next
    	Response.Write("<tr><td> </td><td> </td><td><b>Total pages: " & PageCount & "</b></td></tr>")
    %>
    </table>
    

    Creating compressed ZIP files using VBScript

    Sunday, October 18th, 2009

    SQL/Server database backups tend to compress fairly well, in my experience typically down to around 25% of the original file size. However I discovered backup data compression is only a feature of SQL/Server 2008 Enterprise Edition and it’s not a feature that inspired me to spend thousands extra on a license. I’ve attached a VBScript procedure that I wrote to automatically compress my SQL/Server backups but of course it can be used for any application where you’d like to compress the entire contents of a directory automatically.

    I can’t take credit for the innovative technique of creating a ZIP header in code to make Windows create a compressed ZIP folder. That was a part of a code snippet I found elsewhere quite some time ago. For some reason the MoveHere method of Shell.NameSpace didn’t seem to remove the source file so when move mode is selected I stuck with deleting the source file afterwards. This version displays ZIP compression progress in a dialog box but you could use “ZipFile.CopyHere InFilename, 4″ to disable the progress dialog box.

    ' VBScript to move or copy all files in a folder to a compressed ZIP file
    
    Option Explicit
    Const MoveMode = False
    Const BackupDir = "E:\DB Backups"
    Const TimeoutMins = 10 ' Timeout for individual file compression operation
    
    Sub MoveToZip(InFilename, OutFilename)
    	Dim FSO : Set FSO = CreateObject("Scripting.FileSystemObject")
    	Dim Timeout : Timeout = 0
    	FSO.CreateTextFile(OutFilename, true).WriteLine "PK" & Chr(5) & Chr(6) & String(18, 0)
    	Dim Shell : Set Shell = CreateObject("Shell.Application")
    	Dim ZipFile: Set ZipFile = Shell.NameSpace(OutFilename)
    	ZipFile.CopyHere InFilename
    	Do Until ZipFile.items.Count = 1  or Timeout > TimeoutMins * 600
    		Wscript.Sleep 100
    		Timeout = Timeout + 1
    	Loop
    	If MoveMode and ZipFile.items.Count = 1 Then FSO.DeleteFile(InFilename)
    	Set Shell = Nothing
    	Set FSO = Nothing
    	Set ZipFile = Nothing
    End Sub
    
    Dim FSO : set FSO = CreateObject("Scripting.FileSystemObject")
    Dim Folder : Set Folder = FSO.GetFolder(BackupDir)
    Dim Files : Set Files = Folder.Files
    Dim File
    For Each File In Files
    	If InStr(UCase(File.Name), ".ZIP") = 0 Then
    		MoveToZip BackupDir & "\" & File.Name, BackupDir & "\" & FSO.GetBaseName(File.Name) & ".zip"
    	End If
    Next
    

    Posting to a forum programmatically using Delphi

    Saturday, October 17th, 2009

    I’m a member of a web site where we run various games in the forums and for several that can be tedious to keep track of manually I’ve written some code in Borland Delphi to automate the task. The forum itself is phpBB and I use the web browser component included with Delphi to edit posts directly in the forum. Being able to post to a website as though a user submitted a form is handy for these sorts to tasks where you don’t have access to the underlying forum database. I’ve posted the more interesting parts of the code that format the request and setup the correct headers:

    var
      strData: String;
      PostData, Headers: OleVariant;
    ...................
    strData := 'message=' + HTTPEncode(MessageData) + '&notify=on&mode=editpost&p=' + IntToStr(PostID) + '&post=+Post+';
    
    PostData := VarArrayCreate([0, Length(strData) - 1], varByte);
    
    for i := 1 to Length(strData) do
      PostData[i-1] := Ord(strData[i]);
    
    Headers := 'Content-Type: application/x-www-form-urlencoded' + #10#13;
    
    WebBrowser.Navigate('http://example.com/posting.php?mode=reply&t=' + MainTopicID.Text, EmptyParam, EmptyParam, PostData, Headers);
    

    SQL convert integer to hexadecimal string

    Tuesday, October 6th, 2009

    The following is a small SQL/Server function to convert an integer to a two digit hexadecimal string. I’m using it as part of a GPS mapping application to display the poll type from an Inmarsat D+ terminal.

    CREATE FUNCTION ConvertToHex2
    (
    @pNumber int
    )
    RETURNS varchar(2)
    AS
    BEGIN
    DECLARE @HexDigits varchar(16)
    DECLARE @Digit1 int
    DECLARE @Digit2 int
    DECLARE @ResultVar varchar(2)

    IF @pNumber IS NULL
    SET @ResultVar = NULL
    ELSE BEGIN
    SET @HexDigits = ‘0123456789ABCDEF’
    SET @Digit1 = @pNumber / 16 + 1
    SET @Digit2 = @pNumber % 16 + 1
    SET @ResultVar = SUBSTRING(@HexDigits, @Digit1, 1) + SUBSTRING(@HexDigits, @Digit2, 1)
    END
    RETURN @ResultVar
    END

    SQL/Server function to format integer IP address

    Friday, October 2nd, 2009

    The following is a small SQL/Server function to convert an IP address stored as an integer (actually bigint) into a human-readable string in the format ‘1.2.3.4’. I’m using it as part of a geolocation by IP address project.

    CREATE FUNCTION IP2String (@IPAddress bigint)
    RETURNS varchar(15)
    AS
    BEGIN
    DECLARE @IP1 bigint
    DECLARE @IP2 bigint
    DECLARE @IP3 bigint
    DECLARE @IP4 bigint
    DECLARE @ResultVar varchar(15)

    SET @IP1 = @IPAddress / 16777216
    SET @IPAddress = @IPAddress – @IP1 * 16777216
    SET @IP2 = @IPAddress / 65536
    SET @IPAddress = @IPAddress – @IP2 * 65536
    SET @IP3 = @IPAddress / 256
    SET @IPAddress = @IPAddress – @IP3 * 256
    SET @IP4 = @IPAddress

    SET @ResultVar = CAST(@IP1 AS varchar(3)) + ‘.’
    + CAST(@IP2 AS varchar(3)) + ‘.’
    + CAST(@IP3 AS varchar(3)) + ‘.’
    + CAST(@IP4 AS varchar(3))
    RETURN @ResultVar

    END

    Firefox AJAX requests from different domain / site

    Friday, July 24th, 2009

    Recently I’ve been writing a GPS mapping web page based on the OpenLayers JavaScript library that retrieves GPS data from the server via AJAX. For development convenience I was just storing the HTML file on my local drive for development and fetching the XML data from a different SQL/Server 2008 machine. This worked fine from IE but from FireFox the XMLHttpRequest I was getting a status of 0 (zero) returned and no data.

    It turns out this is a security measure to avoid cross-site scripting which is quite sensible but not that convenient for testing, and in this application the authentication happens at a different level so cross-site scripting isn’t an issue. I found the solution is to add an HTTP header of Access-Control-Allow-Origin: * to the response. Of course you should consider the implications carefully outside of a test environment. As my code is written as a traditional ASP file this just consisted of adding the following line:

    response.AddHeader “Access-Control-Allow-Origin”, “*”