programming panda

i was trying to set up this web application to connect to a HTTP server (happened to be IIS) that is serving some pre-generated report files. The offshore developer that had been working on this had some problems getting it going, so it was left to me to deal with it. The web application was using Apache HttpClient to connect to the http server using some standard http client code below:

<snip>

HttpClient client = new HttpClient();

PostMethod method = new PostMethod(filePath);
method.setRequestHeader("Content-type", "text/html");
client.executeMethod(method);
input = method.getResponseBodyAsStream();
responseBean.setInputStream(input);

</snip>

that resulted in IIS throwing this at us......

The page cannot be displayed
The page you are looking for cannot be displayed because an invalid method (HTTP verb) was used to attempt access.

Please try the following:

Contact the Web site administrator if you believe that this request should be allowed.
Make sure that the Web site address displayed in the address bar of your browser is spelled and formatted correctly.

HTTP Error 405 - The HTTP verb used to access this page is not allowed.
Internet Information Services (IIS)

Technical Information (for support personnel)

Go to Microsoft Product Support Services and perform a title search for the words HTTP and 405.
Open IIS Help, which is accessible in IIS Manager (inetmgr),
and search for topics titled Setting Application Mappings, Securing Your Site with Web Site Permissions, and About Custom Error Messages.

So basically what was happening was that IIS didn't like the POST method that was being used. (HTTP 405 --> Method Not Allowed). The fix? quite simple, use a GET! =) It is easy to forget that POST's primary objective is to post data to the server, not to obtain data from it.

As web devs, we see HTTP 404s a lot...and the occasional 403. it's not everyday you see a 405!

Default location of log4j properties is in /WEB-INF/classes/logj4.properties. There are ways to point log4j to different properties. In spring, we can do something like this.

<!-- log4j setting -->
    <bean id="log4jInitialization"
 class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
  <property name="targetClass"
   value="org.springframework.util.Log4jConfigurer" />
  <property name="targetMethod" value="initLogging" />
  <property name="arguments">
   <list>
    <value>log4j.properties</value>
   </list>
  </property>
 </bean>

where initLogging takes (String location), you can use absolute path or relative path here. But a more elegant way is to use a variable for the application root. like this

<value>${webapp.root}/${log4j.properties.location}</value>

You can get the ${webapp.root} from webAppRootKey. To bring it to the next level we can group all configs into 1 properties file and that will make the build process easier.

The full setup looks something like this

/WEB-INF/web.xml

<listener>
        <listener-class>org.springframework.web.util.WebAppRootListener</listener-class>
    </listener>
<listener>
        <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
    </listener>
<context-param>
        <param-name>contextConfigLocation</param-name>
        <param-value>
            /WEB-INF/applicationContext.xml
</param-value>
    </context-param>
    <context-param>
        <param-name>webAppRootKey</param-name>
        <param-value>webapp.root</param-value>
    </context-param>

/WEB-INF/applicationContext.xml

<bean id="propertyConfigurer"
      class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
      <property name="locations">
      <list>
                <value>/WEB-INF/spring-config/dev.properties</value>
   </list>
   </property>
 </bean>
<!-- log4j setting -->
    <bean id="log4jInitialization"
 class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
  <property name="targetClass"
   value="org.springframework.util.Log4jConfigurer"/>
  <property name="targetMethod" value="initLogging"/>
  <property name="arguments">
   <list>
    <value>${webapp.root}/${log4j.properties.location}</value>
   </list>
  </property>
 </bean>

/WEB-INF/spring-config/dev.properties

#log4j setting
log4j.properties.location=WEB-INF/log4j-config/log4j.dev.properties

/WEB-INF/log4j-config/log4j.dev.properties

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.Threshold=DEBUG
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n

log4j.rootLogger=DEBUG, stdout

/WEB-INF/log4j-config/log4j.prod.properties

# Configuration for receiving e-mails when ERROR messages occur.
log4j.appender.mail=org.apache.log4j.net.SMTPAppender
log4j.appender.mail.To=to@mydomain.com
log4j.appender.mail.From=from@mydomain.com
log4j.appender.mail.SMTPHost=smtp.mydomain.com
log4j.appender.mail.Threshold=ERROR
log4j.appender.mail.BufferSize=1
log4j.appender.mail.Subject=An application error occured
log4j.appender.mail.layout=org.apache.log4j.HTMLLayout

# Standrd System.out appender
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.Threshold=INFO
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n

log4j.rootLogger=INFO, stdout, mail

In the last post I talked about how to parameterize the applicationContext.xml with a properties file. But sometime we would like to access system/context properties in the applicationContext.xml (for example, you want to reference a file in your webapp, but the class only takes absolute path). The spring PropertyPlaceholderConfigurer class would do the work for you as well.

By default, the system property mode is set to use fall back (SYSTEM_PROPERTIES_MODE_FALLBACK), which means if a property is not in our properties file that we supply, the property will be filled with the one in the context/system.

One good use would be getting the application root and use it to reference a config file.

For example

${webapp.root}/WEB-INF/velocity

(In reference to Spring documentation velocity properties chapter)

In order to use ${webapp.root} you will need to setup WebAppRootListener in web.xml

   org.springframework.web.util.WebAppRootListener

and assign it to a context param

   webAppRootKey
   webapp.root

Spring has a very useful class called PropertyPlaceholderConfigurer that helps you to parameterize your applicationContext.xml file. In order to use this, you will need to add the bean in spring like this

 
   
      
         /WEB-INF/spring-config/dev.properties
      
   

where the property location points to the list of properties files.

I use this primarily becasue
1. easier to maintain properties in a properties file than applicationContext
2. switch between different environments, e.g. development vs. staging vs. production

An example of the use of this would be

/WEB-INF/web.xml

   org.springframework.web.context.ContextLoaderListener

    

   contextConfigLocation
      /WEB-INF/applicationContext.xml
   

/WEB-INF/spring-config/dev.properties

# Dev DB Setting
mysql.driver=com.mysql.jdbc.Driver
mysql.url=jdbc:mysql://dev.mydomain.com:3306/
mysql.username=devuser
mysql.password=xxx

/WEB-INF/spring-config/prod.properties

# Product DB Setting
mysql.driver=com.mysql.jdbc.Driver
mysql.url=jdbc:mysql://prod.mydomain.com:3306/
mysql.username=produser
mysql.password=xxx

/WEB-INF/applicationContext.xml

   
      
         /WEB-INF/spring-config/dev.properties
      
   




   
   
   
   

To switch between environments, all you have to update is

/WEB-INF/spring-config/prod.properties

in the applicationContext.xml

To make it fancier, of course you can supply the properties to an ant script when you build your application.

Apache log4j is an excellent logging facility to log application information. It is extremely easy to setup and it is very powerful. It comes with an email appender by default which you can config it to sends email when you log.

Example: Logs message to standard system.out on INFO and Sends email when an application error occurs

# Configuration for receiving e-mails when ERROR messages occur.
log4j.appender.mail=org.apache.log4j.net.SMTPAppender
log4j.appender.mail.To=to@mydomain.com
log4j.appender.mail.From=from@mydomain.com
log4j.appender.mail.SMTPHost=smtp.mydomain.com
log4j.appender.mail.Threshold=ERROR
log4j.appender.mail.BufferSize=1
log4j.appender.mail.Subject=An application error occured
log4j.appender.mail.layout=org.apache.log4j.HTMLLayout

# Standrd System.out appender
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.Target=System.out
log4j.appender.stdout.Threshold=INFO
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n

log4j.rootLogger=INFO, stdout, mail

This is pretty simple. The only setting that controls what to send is the Threshold property of the appender.

To map the URL root to certain welcome file, we normally just specify the welcome-file in the web.xml like this

index.html
index.jsp

What this means is, if in a directory without specifying any file name, the web application will first look for index.html and then index.jsp.

Now add struts framework into the picture, default struts extension for 1 and 2 is *.do and *.action respectively.

There are 2 ways of making our application pointing to the special struts action.

#1: Meta redirect
You can keep index.html or index.jsp as welcome-file and what you need to do is put a meta refresh in these index files

Struts 1

meta equiv="refresh" content="0;URL=/homepage.do"

Struts 2

meta equiv="refresh" content="0;URL=/homepage.action"

But the downside of this is - whatever action name you use would appear on the browser. (i.e. www.mydomain.com always appear as www.mydomain.com/homepage.action even user types in www.mydomain.com)

#2: Change welcome-file and add dummy place holder file

The second way (or better way) of fixing the struts action mapping to the URL root is to update the welcome-file list.

Struts 1

homepage.do

Struts 2

homepage.action

And then in the WebContent Root, add a dummy file (e.g. in Struts 2 you need an empty file homepage.action)*This step is important, without the empty dummy file, it won't work, at least in Tomcat*

Now when you hit http://www.mydomain.com your application will hit the homepage.action directly without adding the homepage.action after your url.

I have been having problem in setting up Ubuntu on my Dell Workstation. Ubuntu documentation provide minimal help so as the forums. And I finally got this

http://albertomilone.com/nvidia_scripts1.html

The script auto detects your video card and install proper driver.

And of course before you run this, I recommend you to uninstall all your drivers and refresh your X.11 setting to default.

GAE(Google App Engine) is a very high security environment, which google limits you in everything. In order to setup your struts 2 on GAE properly, you will need to change the OgnlRuntime security manager.

This is the error you will get on browser

HTTP ERROR: 404

result 'null' not found
RequestURI=/

And on your log

Jul 20, 2009 3:37:20 AM com.opensymphony.xwork2.util.logging.commons.CommonsLogger error
SEVERE: Unable to set parameter [location] in result of type [org.apache.struts2.dispatcher.ServletDispatcherResult]
Caught OgnlException while setting property 'location' on type 'org.apache.struts2.dispatcher.ServletDispatcherResult'. - Class: ognl.OgnlRuntime
File: OgnlRuntime.java
Method: invokeMethod
Line: 508 - ognl/OgnlRuntime.java:508:-1
 at com.opensymphony.xwork2.ognl.OgnlUtil.internalSetProperty(OgnlUtil.java:392)
 at com.opensymphony.xwork2.ognl.OgnlUtil.setProperty(OgnlUtil.java:143)
 at com.opensymphony.xwork2.ognl.OgnlReflectionProvider.setProperty(OgnlReflectionProvider.java:91)
...
...

What you have to do to fix this problem is to add a listener to set the security manager to null.

Listener:

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import javax.servlet.http.HttpSessionAttributeListener;
import javax.servlet.http.HttpSessionBindingEvent;
import javax.servlet.http.HttpSessionEvent;
import javax.servlet.http.HttpSessionListener;

import ognl.OgnlRuntime;

public class ONGLFixListener  implements ServletContextListener, HttpSessionListener, HttpSessionAttributeListener {

   public ONGLFixListener()  {
   }

   public void contextInitialized(ServletContextEvent servletContextEvent)  {
       OgnlRuntime.setSecurityManager(null);
   }
  
   //Empty methods that do nothing for the interfaces

web.xml

  com.example.ONGLFixListener

This should fix the problem. =D

My co-worker asked me how big is the mysql DB, I don't really know how so I did a little research and found this...

SELECT table_schema "Database", sum( data_length + index_length ) / 1024 / 1024 "Data Base Size in MB"
FROM information_schema.TABLES GROUP BY table_schema ; 

And it works =D
It basically just go to the information_schema and sum up the size of the tables of each schema.

The output looks something like this

+--------------------+----------------------+
| Database           | Data Base Size in MB |
+--------------------+----------------------+
| dms                |           4.51943684 | 
| hrform             |           0.06588364 | 
| information_schema |           0.00390625 | 
| mysql              |           0.42193222 | 
| ptrng              |           0.05598068 | 
| securewiki         |           1.52050400 | 
| wiki               |         584.47095490 | 
+--------------------+----------------------+

Not sure why tomcat doesn't have .iso and .msi mime types in web.xml. And smart Microsoft IE doesn't do a great job in file download without proper http header, it actually opens .msi and .iso and display it. In order to help IE download the binaries probably.(Note that, it works fine on Firefox and Safari). You have to explicitly add the mime type in your tomcat. To do this, simply add

    iso
    application/octet-stream
  

in the $CATALINA_HOME/conf/web.xml (or your webapp's web.xml)

We all enjoy the logging facilities in the modern programming language (e.g. log4j in java, c# logging in c#)

but what about shell script? The answer is an one liner

LOG(){echo "$(date +%c) $*"}

for console and

LOG(){echo "$(date +%c) $*" >> log.txt}

for txt out put.

Of course this is not the as sophisticated as the logging framework like log4j, but it simple and easy to use.

To use the function simply calls

LOG "Message"

There are a lot of benefit of using apache http server as a front end of an application server. One of the reason is security. Linux doesn't allow you open port under 1024 for a non root user.

And in some cases it is a good idea to use apache as your front end anyway (e.g. you want to use apahce for load balancing or as a proxy server, or take advantage of a very stable http server)

I personally use apache as a front end for plone+zope and tomcat. Here is a brief description on how to run apache in front of a zope+plone setup

When you have your zope+plone setup, you will access the plone instance like this
http://myhost:8000/Plone
or in general
http://[host]:[port]/[name_of_plone_instance]

and ZMI should be running here:
http://myhost:8000/manage

In order to hide plone+zope behind apache, you will have to add this into your apache config file

RewriteRule ^/(.*) http://127.0.0.1:8000/VirtualHostBase/http/%{HTTP_HOST}:80/Plone/VirtualHostRoot/$1 [L,P]

where %{HTTP_HOST} is a apache variable which will be automatically filled in.

The first part of the rewrite rule

RewriteRule ^/(.*) http://127.0.0.1:8000/

What it does is to tell apache to route all the port 80 request to localhost port 8000 which zope is running.

The second part of the rewrite rule

VirtualHostBase/http/%{HTTP_HOST}:80/Plone/VirtualHostRoot/$1 [L,P]

is to tell the VirtualHostMonster in zope to map all the url in the page to port 80 even it is on port 8000

So now when we hit the url http://myhost/ it direct all the traffic to the underlying plone instance at port 8080.

At this point, everything should work nicely. But what if we want to go to access port 8080 for the ZMI. If you have port 8080 open, you should be able to just hit it on your favorite browser. What if we do not want to poke a hole on the firewall. What you can do is to do a port forwarding over ssh.

ssh -L 9999:localhost:8080 myhost

What it does is open a secure tunnel and forward all the traffic from the remote host to your local box.

In general, you will do

ssh -L [local_port]:localhost:[remote_port] [remote_host]

So when you hit the URL, http://localhost:9999/manage you should be able to see the ZMI in the remote host.

For security reason, Linux doesn't want you to run your stuff on port 80 except apache. And in some cases it is a good idea to use apache as your front end anyway (e.g. you want to use apahce for load balancing or as a proxy server)

I personally use apache as a front end for plone+zope and tomcat.

When you have your zope+plone setup, you will access the plone instance like this
http://myhost:8000/Plone
or in general
http://:/

and ZMI should be running here:
http://myhost:8000/manage

In order to hide plone+zope behind apache, you will have to add this into your apache config file

RewriteRule ^/(.*) http://127.0.0.1:8000/VirtualHostBase/http/%{HTTP_HOST}:80/Plone/VirtualHostRoot/$1 [L,P]

where %{HTTP_HOST} is a apache variable which will be automatically filled in.

The first part of the rewrite rule
RewriteRule ^/(.*) http://127.0.0.1:8000/

What it does is to tell apache to route all the port 80 request to localhost port 8000 which zope is running.

The second part of the rewrite rule
VirtualHostBase/http/%{HTTP_HOST}:80/Plone/VirtualHostRoot/$1 [L,P]

is to tell the VirtualHostMonster in zope to map all the url in the page to port 80 even it is on port 8000

So now when we hit the url http://myhost/ it direct all the traffic to the underlying plone instance at port 8080.

At this point, everything should work nicely. But what if we want to go to access port 8080 for the ZMI. If you have port 8080 open, you should be able to just hit it on your favorite browser. What if we do not want to poke a hole on the firewall. What you can do is to do a port forwarding over ssh.

ssh -L 9999:localhost:8090 myhost

What it does is open a secure tunnel and forward all the traffic from the remote host to your local box.

In general, you will do

ssh -L :localhost:

So when you hit the URL, http://localhost:9999/manage you should be able to see the ZMI in the remote host.