RSS
 

Archive for the ‘Java’ Category

Application Deployment – web and other with php-packager

30 May

Introduction

Deployment of applications on a server is always an issue for developers of applications that need to be installed in some way. Applications like this include specifically web applications built with or on top of Drupal, SilverStripe, Symfony or other PHP web application frameworks. But this is not limited to PHP. Linux server applications can also be Java EE or other JVM applications that run on a server or even C(++), Python, Ruby (on Rails) and any other software. There are usually files that have to be generated or/and copied from a development machine to a test machine and ultimately to a production machine. This process is usually error prone and tedious and needs a lot of documentation. With good documentation this can be reasonably easy, but it is never trivial and humans make mistakes. On top of that one often needs to compile ones own or third party applications and so on, because there is no package for that software yet, because it is far from trivial to roll your own packages. Whether that is a C++ application or a Java application doesn’t really matter. However, we’ve learned that compiling your source code on a production machine is really bad practice, despite it being the easiest way to do things. The main reason for this being bad is that this implies that there is a compiler on a production machine. This is a security risk because if an attacker gains any kind of access to the machine, they can compile their own software on it and run it at their will and possibly compromise the production machine.

However, to compile and/or deploy ones applications there are usually a series of steps that can be bundled into a script. For a while I was using deployment scripts so that I could track a version number within git tags. These deployment scripts were individual to specific applications and never easy to write. Some stuff was generic and other stuff very specific to the application.

PHP-Packager was born

So, I’ve been conversing with my colleague about how we should go about deploying our applications in a repeatable manner. While there are things like Juju, Puppet, Maven, Gradle and so on, we are using mainly Linux as a platform for servers and specifically a Debian based distribution you might have heard of called Ubuntu. As you might know Ubuntu and other Debian based distros (actually also Red Hat/CentOS/Fedora as well) manage their software with packages and ultimately with APT or YUM and the like. For Ubuntu and other Debian based distributions like Linux Mint these are usually .deb packages with dependencies and install/uninstall scripts. For Red Hat based distros, this is usually .rpm.

Finally, we decided to make our own APT repositories for all our VMs, even though we had feared and avoided it in the past. One test and one production repository. Technically, you can make a development repository as well, but for now that would be overkill. For that we needed a mechanism to package and deploy our applications to the repository while still keeping track of versions in our source code repository so we could roll back to a known version. For version control we use Git and for that reason we decided the current Git commit should simply be tagged with the current version over which we have some power, but we don’t want to keep track of every minor version manually.

There is already a solution out there to make packages. It is called FPM, by jordansissel (https://github.com/jordansissel/fpm). It greatly simplifies the process of making packages for all major distributions. However, since I am most comfortable with PHP as a scripting language and not Ruby, Python or even Groovy, I came up with the idea to wrap it with PHP and some little convenience automation, so I don’t have to do the generic stuff. PHP is a powerful, object oriented language and is one of the easier high-level scripting languages. It is itself open source and free and has many other advantages. The name seems to imply that it is only for the web, but apart from the opening <?php tag there is not much more that enforces it to be used only for web apps. It can easily be used from the command line as well.

PHP-Packager or packager for short, is a little script/application that allows for as-easy-as-it-can-get deployment of software via packages. Packages can have dependencies and basically bundle a set of files and scripts triggered by events/actions.

PHP-Packager requires one folder and at least one file within your repository. You can read more details on how to use it on its Github page at https://github.com/geekdenz/packager.

There is more potential for it like more generic tools which one might need in the deployment process, but currently it has the following features:

  1. Package files into a bundled deb or rpm package.
  2. Add after-install and before-remove php scripts for custom procedures like setting up a database for an application or creating symlinks or other files.
  3. Tag your git repository with a version tag. The version is kept track of and can be changed at any time in version.txt in the root of your application’s source code.
  4. Automatically upload the deb or other package to a repository with scp.

There are probably many other things that could be simplified or improved. But this is the current state of this application.

 

Which Technology To Choose?

03 Jan

Since this seems to be interesting for a lot of developers, I thought I’d share my thoughts:

Introduction

Which technologies to choose is getting increasingly more difficult for developers, web and embedded alike. Questions like “Why Java?”, “Why HTML 5 (HTML + JavaScript + CSS Modern Extensions)?’, “Why C++?”, “Why PHP?”, “Why Python?” or “Why not something else?” are not easy to answer. There are many advantages and disadvantages in using any of these programming languages and environments. Within these languages there are even more possibilities of choosing development tools called frameworks. When talking of frameworks, I mean jQuery (JavaScript), OpenLayers (JavaScript), (Geo)Ext (JavaScript), Twitter Bootstrap (HTML5), Drupal (PHP), Symfony (PHP), Java EE, Wicket (Java), Hibernate (Java), GWT (Java -> JavaScript), just to name a few I’ve had the pleasure (and pain) to work with. More and more I’m coming to the following conclusions:

Choosing The Path Of Least Resistance

Maximum Gain With Minimum Pain

This has always been my main goal with the development tools I choose. After all, computers are there to make our lives easier and not harder. You might be surprised of some of the choices I’ve made with OS and tools. While choosing the Windows or MacOS platforms seems like the easier path, Linux and yes, the command line, actually have proven easier or at least more powerful by now. So, what seems the easiest at first might be harder in the long run and vice versa. Also, Linux tooling for web or any kind of development is far better documented, supported and a lot cheaper (free). So, if you are like me (and most other people) and have more time than money, it is probably worth your while to learn and invest (time, at least) into Open Source. It has had a great return in terms of moral and now even monetary rewards. There are companies out there that invest their staff’s’ time and resources into Open Source. Among those not least Google and Facebook, the biggest Internet companies in the world!

Frameworks

Frameworks might in fact make life harder rather than easier. Sure, I use many framework in my work, but sometimes it seems like learning and choosing a framework sometimes takes longer than writing your own and using it. At least attempting to write your own framework(s) has proven as a good exercise for learning how to use them and vice versa, choosing a framework and using it might give you ideas on what to do and more importantly on what not to do. Now, I haven’t written my own complete framework as yet, but I have started doing so. I guess once you start your own framework, you are never finished. The main framework I am working on since about 2 years has been a client side spatial portal application framework called the LcrToolkit (Landcare Research Toolkit, used on http://ourenvironment.scinfo.org.nzhttp://smap.landcareresearch.co.nz and others to come out soon). Thinking back now, I could have come up with a more original name for it, but that is just a side note. It is a very specialised framework for putting up spatial portal applications. Learning to use a framework is a big time commitment, whether this is Microsoft .NET, PHP (Symfony, Drupal, …) or Java’s many frameworks. So, if it were for saving time, some people say, use .NET, because some people believe the ads that the time investment will be minimal. This (in my opinion) is wrong! Learning PHP and JavaScript frameworks can be achieved much faster, because these languages (at least the basics) are very simple AND they are openly developed with tons of documentation and since they are free, there is a lot more support from developers and advocates. Not only do these developers innovate and support free products because they are super nice people and want to make the world a better place, they develop Open Source products, because these products live longer and if they are good, attract a lot of developers. This is not only because Open Source is better, but also because developers get credit for what they have achieved. Furthermore, Open Source is very competitive and therefore secure. To get your name out there as an Open Source developer, you have to be (one of) the best. That means you have to compete, BUT always in a friendly and respective manner. To be the best, you have to be prepared to fail to succeed. You have to be open to being corrected and that is not always easy to handle. To get back to Frameworks: There are a lot of them out there and I have named a few already. It seems like there are simply too many. The more I research frameworks, the more time I could spend trying them all out. The possibilities are endless. So, one must put a vigorous filter into ones research. Lately, I’ve been wanting to implement a testing framework inside the LcrToolkit and eventually went with qUnit (JavaScript), not because I think it is the best, but simply because I do not have time to try them all. qUnit is simple, well documented and works just fine. Surely, there must be more advanced testing frameworks for JavaScript, but qUnit is easy to pick up and that is the main criteria these days for me. Things like Hibernate and JPA don’t really appeal to me. They are bloated, take a lot of time to learn and don’t really gain you anything. The only benefit you get from ORMs is that you are “independent” of a particular database management system. Just use PostgreSQL for Christ’s sake. It is the most advanced database out there, and yes I mean the most advanced DBMS. It is superior to even Oracle and far better than MySQL or Oracle’s DBMS. Not that Oracle is a bad company… anyway, I’m missing the point. Frameworks: There are so many that one cannot grasp all of them. I’ve been a bit of a grasshopper on frameworks. I’ve worked with Drupal, Joomla, CiviCRM, GWT, Wicket and all of the above and have taken away that frameworks are great, BUT cannot replace good developers. You need to be thinking all the time about security, look and functionality of your app, whether it is a web app, website or even “just”  an embedded program.

To be continued… ??

 

Howto: Install Apache Tomcat 6 on Ubuntu 11.10

21 Feb
  1. Download Tomcat 6 from http://tomcat.apache.org/download-60.cgi or a mirror listed there.
  2. Extract it in the /usr/local directory:
    cd /usr/local/
    sudo tar xvfz ~/apache-tomcat-6.0.35.tar.gz # or wherever you downloaded tomcat to
    sudo ln -s apache-tomcat-6.0.35 apache-tomcat-6 # or whatever your tomcat version is

This post is incomplete, but might get you going. More docs can be found @ http://www.eclipse.org/forums/index.php/t/272457/

 
No Comments

Posted in Howto, Java, Linux