Keith's Web Blog RSS Feed

PHP Installation

Thu, 12 Feb 2004 13:29:23 -0700

i generally create a small install script for my php installs: #!/bin/sh APACHE=/usr/local/apache APXS=${APACHE}/bin/apxs DB=/opt/databases MYSQL=${DB}/mysql PGSQL=${DB}/pgsql OPENSLL=/opt/openssl PHP_BASE=/usr/local/php VERSION=5.0.3 PHP_HOME=${PHP_BASE}-${VERSION} ./configure -prefix=${PHP_HOME} \ --with-apxs=${APXS} \ --with-openssl=${OPENSSL} \ --with-zlib \ --with-dom \ --with-xmlrpc \ --with-pgsql=${PGSQL} \ --with-mysql=${MYSQL} \ --enable-mbstring \ --enable-mbstr-enc-trans \ --enable-soap make make install cd /usr/local ln -s php-5.0.3 php cp /php.ini-dist /usr/local/php/lib/php.ini note: be sure to change the value of your php directory when copying the files over. I use this as a kind of standard. Also, you have to edit your httpd.conf file to tell apache to load certain extensions with php. add (after mod_negotiation.c): # # And for PHP 5.x, use: # AddType application/x-httpd-php .php AddType application/x-httpd-php-source .phps

Wrappig a postgresql transaction

Sun, 29 Feb 2004 01:55:15 -0700

Some code for wrapping a postgresql transaction: pg_exec($conn, "begin work"); $res = pg_exec($conn, $sql); if (!$res) { $err = pg_errormessage($conn); pg_exec($conn, rollback"); return 0; } else { pg_exec($conn, "commit"); return 1; } I'm assuming that this wrapper code is inside of a function (hence the return). I prefer placing this code in a reusable object which allows me to handle transactions better and so that other objects can extend this base class. It also makes maintaining the connection string easier since you can set it in the base constructor class and not worry too much about setting it later. I've simplified this although if you want your application to be smart, you can't do this in such a generic manner.

Email Validation Regular Expression

Sat, 01 May 2004 08:38:53 -0600

Here's a quickie for validating email in javascript and php (note: i added lines to make it more readable than copy-and-pastable) javascript: var email_exp = /^(([^<>()[\]\\.,;:\s@\"]+ (\.[^<>()[\]\\.,;:\s@\"]+)*)|(\".+\")) @((\[[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\])| (([a-zA-Z\-0-9]+\.)+[a-zA-Z]{2,}))$/; php: if (eregi("^[_a-z0-9-] +(\.[_a-z0-9-]+)*@[a-z0-9-]+ (\.[a-z0-9-]+)*(\.[a-z]{2,3})$", $email)) { *DO_SOMETHING* }

Smarty Plugins and Objects

Mon, 23 May 2005 02:26:55 -0600

One massive complaint I have is that there's no inherent mechanism for re-use within the Smarty plugin structure. With, for instance, Template Toolkit, you inherit from the template parent class to create a new plugin. The benefit of that is that you can re-use and expand on base classes that attempt to perform larger scale functionality, like a calendar object. However, there is an easy way around it: adding classes in the Smarty plugin directory and using functions as wrappers to those objects. By doing it that way you can easily create your own reusable plugin objects and having the functions acting as factories or wrappers around them. Actually, this is a slick way of condensing your view code since you don't have to do a ton of setup, except in the parameters section of the function call in the template.

bitching about php

Mon, 02 Jul 2007 10:59:30 -0600

i like php but it does need better syntax to be more competitive against other syntactic sugar style scripting languages like Perl or Ruby. for once, rather than relying on a million functions for handling what should be more embedded capabilities. for instance, arrays and associative arrays really need to be revamped. it's something that i really like about Perl in how hashes and arrays having some qualities that allow for nicer formatting of code.

another thing that i think PHP should do is remove interfaces and go with multiple inheritance. i really see no point in PHP using interfaces. I find interfaces to be useful in heavily typed languages. in the case of Java, for instance, interfaces work because you need to specify a return type and the elements in the interface must be specific. Interfaces in PHP to me have no meaning because you're just being forced to implement a function with a meaningless signature. the signature can contain anything without enforcement. so how can you utilize that as an ADT? you're really better off with an abstract class at this point. if PHP wants to improve the use of interfaces, i suggest enforcing a return type and making the signature of the interface heavily typed. otherwise, you'll never see someone like me employ an interface directly in his code.

Multiple inheritance for me is more important. i hit a situation the other day where i could've used multiple inheritance. i had code duplication in two classes that performed similar functions. i would've encounted the diamond inheritance problem because of the way i would've inherited in this case. however, the only way to prevent this duplication of code would've been to do an inline require_once statement. still not really an object-oriented way to go about solving a problem!

Java Is NOT An Agile Language

Tue, 07 Aug 2007 09:40:50 -0600

My big boss asked me why my side of the project has been taking so long. Why would a business layer be so hard to code up? Why is the testing ridiculously slow?

My answer: Java doesn't help this.

Perhaps, it's my level of expertise in the language, but after dealing with this project, I've come to realize just how HEAVY Java can be. I haven't had a chance to work in detail with the latest version, so I can't say that some of the newer features have made it better (I'm sure it has, but there are still deficiencies in the language). It could be my lack of experience with Eclipse. Maybe I'm not utilizing the right plugins and doing far too much "hand coding" in terms of boilerplate code.

No, the problem is that the language is too bulky and takes itself too seriously.

Read that statement again. Especially the last part.

Let me provide an example.

Let's say you have a primitive type that's supposedly a char and is saved into the database as a number (0 or 1). so in reality you need an int. But let's say another object requires it either as an int or be turned back into a string. So if you try to parse it, you might get an exception rather than letting the language auto-convert it. So you're doing many things here: 1) checking if the value is null; 2) checking if the type can be converted; 3) potentially converting it over to another type; 4) converting it again. And you may have to wrap all these operations in a big try/catch block. If you're working with a database that may contain null values, and you're forced to use primitive types, you're royally screwed. I mean, all you're trying to do is guarantee a value type, but you still need hundreds of checks.

This is just awful. And it's just for one value.

What if you have 40-50 values? How much of this do you have to deal with? What if you have to deal with several operations that require you to do this 40-50 times? What if there's no easy way to loop through the parameters in your object without having to take out the equally bulky Reflection API?

In perl, since objects are glorified (or blessed more specifically) forms of hashes, arrays, scalars or globs, you could, especially if you use a hash, provide a method that can loop through, utilize the Fields pragma, and easily get the same benefit.

In PHP, you can utilize variable interpolation by doing something like this for a dynamic method call:

$obj->$method()

You're pretty much out of luck with Java (at least in doing some easily the same way). You can't naturally take advantage of in-built features to really help you through these situations.

Worse yet, you might be forced to code declaratively in XML because it's nearly impossible to do something easily and dynamically in Java. Take Spring and Hibernate as examples of this.

Out-of-the-box, Java is just a monster of a language. You are forced to employ various plugins, IDEs, frameworks, etc. to really get any significant and immediate benefit. And that can take a while to do as well.

Notice the words used there: "a while." That implies time and implies not moving fast, which in turn implies lack of agility.

More of my friends are turning towards languages like Ruby and being empowered from getting more work done through frameworks like Ruby on Rails. I think people who continue to use Java are quite arrogant in believing too much in Sun's marketing. Java only is good in the hands of those that use it. But for me it's too difficult to get any real benefit that I can see.

The truth is, at the end of the day, you just want to be productive. The language is truly irrelevant as long as you and your peers utilizing it understand in detail how to get the best out of it. I for one see Java as being verbose, cumbersome, and unnecessarily detailed. The flaws that kept a lot of C/C++ programmers away I think never were truly addressed. C/C++ programmers have some advantage over Java people since they have more power and speed naturally.

Overall, if you need to execute something quickly, consider NOT using Java to do the job, unless your people are extremely adept at it. Mine aren't, nor am I so I think I will continue to stay away from it where possible.

static data members and inheritance in PHP

Tue, 09 Oct 2007 18:23:34 -0600

i found something interesting the other day using PHP. let's say you have a base class (which would be using the singleton design pattern) with a static data member called $instance. later, you inherit the base class with another two additional classes and have them utilize the $instance data member (say through setting it in a method called getInstance()). originally, i had thought that each instantiating class would get its own copy of $instance. nope!!! because the data member is static in the base class, it really only pertains to the base class, especially when you start referring to it as self::$instance.

i'm not sure if this is a good explanation of the situation, but i learned a truly valuable lesson today: be careful when using static data members and inheritance. If you intend to do some sort of polymorphism with the static data member, it probably won't behave as you'd expect.

Zend Framework

Wed, 10 Oct 2007 08:44:01 -0600

I'm currently hacking away at the Zend Framework, which is probably my one millionth framework (okay, maybe not that many. I think I'm up to five or six now). While it's got a lot of jazz, the thing I've come to realize is that these frameworks are nothing more than a bunch of hammers, nails, boards and glue lying around. You still have to draw your own plans to get a basic house going. Or in our case, a real workable website.

The thing I hate about some of these frameworks is that despite preaching good practices by componentizing everything (thus decoupling code), they never actually attempt to enforce on their users true good practices. For instance, the controllers provided by Zend can end up holding monolithic code and redundant. You can dump all types of business functionality within one controller and then potentially copy the same functionality into other controllers. Or your controllers may not necessarily follow the same programmatic flow that one may desire.

It's surprising that these frameworks are not more strict considering the depth of knowledge of web programming that has been developed in the past 10+ years. For instance, the main Zend action controller has a method called preDispatch which would allow one to place authorization and validation code that might be common between controllers. Well, considering that most controllers require authorization and validation code, why not just house part of that code inside of there in the first place? You're going to end up subclassing this class regardless just to create a base class that can be common between controllers.

Another truly annoying "feature" (or lack thereof) is the authorization and authentication methods that come with Zend. For authentication, you practically have to write your own adapter and wrapper just to get a simple login to work. If you want to use it along with a PHP session, you then have to draw together those components on your own. They only provide a thin wrapper just above the $_SESSION global, which still can be accessed. I don't really see a point.

Authorization is handled a little better, but you still have to provide the hooks. It's far too barebones to really be useful. You can employ it just to say that you're utilizing that piece of the Zend framework, but you're honestly better off just rolling your own authorization scheme.

The most annoying thing is that there's no out-of-the-box login controller nor controller that houses the authorization logic. There's no real hooks that allows one to tie their controllers easily into Zend's ACL components. You're practically on your own to put in something that probably has been done a billion times in very efficient manners. This is horrible.

Next we come to the validation piece. I've rarely seen validation done well. The Zend framework is no exception. It's nearly as bad as the Spring Framework's validation methodology. The problem is that you cannot easily do declarative validation. I basically want to say given a page, here are several fields and for each field, here is a set of validations with possible values to validate against. The Jakarta Commons Validator does a much better job, except the XML validation is a pain to use. Still that's the basic gist, combined with using something like Struts to automatically execute the validation piece. On the other hand, in Zend, you're expected to put it in the preDispatch method. I mean, if you already know you're going to expect validation to occur prior to an action, why not just strategically have a subclassed controller that comes with Zend handle the validation automatically? Although I think the Spring Framework's controllers suck for the most part, the one thing they did do correctly was provide numerous out-of-the-box controllers for various purposes. Zend only provides two.

The Model layer was probably the only piece that was done halfway decently. It's fairly easy to program and nothing I haven't seen before. Considering the popularity of some frameworks like CakePHP, Ruby On Rails, and Symfony, I'm quite surprised that Zend left out the scaffolding code. I avoided Symfony because I thought Plone and its scaffolding code was not handled very elegantly (for instance, forcing one to use their convention; legacy databases wouldn't be compatible). Still though, scaffolding is very important in this day's competitive and overdone framework laden programming community. It's shocking to see such a useful tool ignored (at least for now).

Again, my biggest complaint is that Framework only organizes the more web specific code from PHP into a more organized set of classes. I think Frameworks like these need to impose even stricter interfaces and have more useful tools or objects that are well known so people can concentrate on the application logic rather than writing additional wrappers to improve the convenience of using some of these frameworks.

new job update

Thu, 25 Oct 2007 10:49:59 -0600

I won't give out the name of my company right now. i really want people attempting to contact me about job offers, etc. to stop unless they can provide a solid deal for something like $2 million without me doing anything. That's my current price tag. No less.

There that should put off people.

But I will talk about the place.

Unlike my last place (which I'm attempting to erase from memory), my new spot has been quite good so far. I'm certain there's a high expectation for output from me, but I've got my own deck of cards that I like to draw from when it seems like people have an Ace-King against me. Right now, I'm being "encouraged" to speak more Japanese. For instance, today's lunch was almost completely Japanese. I figure staying in this environment for a few years will pump up my Japanese level to near fluency. Hopefully, I can get up to at least 2-kyu level in a year.

I've been incredibly motivated as of late. No life draining meetings. No sitting in endless discussions on what we INTEND to do, but rather taking action IMMEDIATELY to remedy a problem. No monstrous hiearchies of incapable people. No stupid vendors mind controlling weak willed, techno-illiterate management. Just solid, hard core work, coding and learning.

Just in terms of sample things I've been working on:

Extending the Zend Framework and gluing all the loose pieces together.
Utilizing Phing to automatically generate projects, including checking out and importing newly created projects from and into Subversion
Setting up my own Subversion repository with Webdav access
Setting up Samba for Windows access on my local machine
Expanding my Javascript/AJAX knowledge through implementing JQuery (and perhaps MooTools as well in the near future)
Learning a new Japanese XML filesystem query tool called Musashi
Writing PHP wrappers around the Musashi command line, utilizing the Command object and Chain of Responsibility design patterns
Localizing the front end with Smarty's config files.

I'm hoping to re-build or convert over the log files into a binary format that's faster and easier to read and replace the Musashi command line interface sometime. I'm thinking of implementing C/C++ for raw speed, perhaps figuring out how to employ either that or something like Berkeley DB to generate flat files and improve performance overall.

Lots of fun stuff. Also, I plan to build tons of in-house tools, stuff I've been dreaming of for years. I have a lot of freedom in terms of technology, so I pretty much get to do whatever I really want. I don't need a 20% type of rule for my work. I'm doing what I want at 100%. Nice, ain't it?

The other thing I'm doing is learning about a "real" Japanese environment. My previous company was becoming one, but in a negative sense. The current place has bits and pieces of the Japanese environment such as saying "Ohayogozaimasu" in the morning and "Otsukaresamadesu" as you're leaving. Also, many people work late. I plan to inject some California corporate culture into this place though. I bought a bunch of Krispy Kreme donuts tonight and will start making my rounds in my "political capital." I will be loved in the morning (and probably hated by late after since everyone will most likely be clutching their stomachs as I did in pain tonight!!!!!)

But this is truly one of the first almost "pure" Japanese environments I've ever worked. Everyone is Japanese here, except me. I'm practically a rock star in some ways. Since I know English and technology, I have a great deal of potential value to add to people, especially considering that most people want to learn more English. I've offered my services as an English teacher for free (and I enjoy doing this just to show that there are people in this world that don't need to make a pretty penny unlike those Nova greedy corporate pigs).

What I'm attempting to do most of all is really get into the minds of a Japanese business. Of course, we do have foreign influence since the company is mostly owned by the parent in America. However, most of that influence feels distant. What's nice though is that you don't have a bunch of foreign idiots in the local office screwing with people with their lack of understanding of the Japanese culture and business methodology. Okay, I am here but I don't attempt to disrupt and overburden people with my bumbling behavior. I try to inject productivity, encouragement and positive feelings as much as I can with my coworkers.

Compare that with my previous company where I saw these foreigners, mostly East coast people, who just couldn't run the damn show. And still can't. It's embarrassing. I couldn't tell if the problem was that of the industry (i.e. insurance) or the fact that these people were just plain dumb. But it was utterly pathetic watching these people get twisted left and right, but having enough power to screw things up because they couldn't properly assess the situation. I truly pity all the workers at the bottom attempting to scrape by in such a company. It didn't feel like a real company at all. More like a drama being staged for a reality show (or a Surreality Show).

Nonetheless, at my current place, I really feel like I'm part of something. Sure, our desks are small and there are notions of appropriate etiquette in the office. And I'm certain that some time down the line I'll be asked to perform in higher levels as a Japanese would. But in those cases, I suppose it's up to me to mature and evolve. That ought to truly increase my value, even though I think most companies in America won't ever whollistically understand my overall intrinsic value (I'm NOT JUST A PROGRAMMER DAMN IT!!!!!)

Overall though, I'm fearful of saying it but I feel like I've found my calling. I hope that this situation last and can improve even more.

PHP Associative Arrays and Numbers as Keys

Wed, 14 Nov 2007 23:31:46 -0700

I remember having this problem before and I spent a little time struggling trying to figure out this issue. The thing is that if you want to use integers as keys for an associative array in PHP, you need to quote them. That is, when you're generating a new key. Otherwise, you'll receive a warning:

Warning: Illegal offset type

It's very annoying.

Skilling Up!

Tue, 20 Nov 2007 09:24:04 -0700

Right now, I'm teaching myself some ruby. I've mucked with it in the past, but I've decided to pursue increasing my python, ruby, c/c++ skills. i'm pretty satisfied with my php skills now. i just need to figure out how to use CakePHP eventually and I'll be set. With ruby, I just got through basic inheritance and some simple string manipulation. Syntactically, I like Ruby more than PHP, but I'm extremely used to coding in PHP now. However, the syntactic sugar of Ruby is slowly drawing me in. Lots of shortcuts rather than elongated function names. I hope to get to Rails sometime in the next few months. Eventually, I'd like to put together a simple Rails application just for fun. Maybe I'll build my own music/movie organizer AJAX website. The same with Python in terms of getting into it for fun. I'm not a big fan of Python's syntax compared with Ruby, but Google's use of it along with Zope look like compelling reasons to give it a try.

I'm not as sure what to do with c/c++ in terms of a project. I always wanted to participate in building some KDE project. Not sure what yet.

Got some cool chances to work with a few new toys at work. Got Samba to work (finally!) It's nice being able to go between my Kubuntu system and my laptop running XP Pro and copying files over transparently.

Setup a Subversion repository with WebDav. I want to get a better bug track system or something that'll tie into Subversion. My friend suggested a site for this, but I don't like the fact that they use their own login system. I want a centralized login system that utilizes the same data. Don't want to have to hack a system for this purpose.

Also, worked a bit with Phing. Phing is like Ant for PHP. Only they're still generations behind in terms of support. I had to write a bunch of my own plugins. But that's the cool thing with it. Maybe I might contribute to Phing in terms of adding my plugins to their repository.

Lastly, I've worked with this gzip'd compressed XML data unix set of command tools called Musashi. It was a major bitch to set up. But I wrote wrappers in PHP to be able to interface with the tool. Not native, but still good enough for now.

Request From PHP: We Need An init() method!

Sun, 25 Nov 2007 00:02:26 -0700

One advantage of mod_perl and servlets over PHP is the fact that they have the ability to pre-load information on startup of the web server. Right now, I don't know if PHP has an equivalent but it's something that I'd like to see happen. The reason is that preloading allows some performance optimization by putting some processes or data structures into memory so that the users hitting the server won't have to take a hit. It's kinda like a cache of sorts. I do think that PHP keeps some parts in a cache in memory before eventually cleaning itself up. But you don't want something like this in all cases. It would be a very nice feature.

Unfortunately, in terms of hosting, this feature probably wouldn't be allowed since you really need a dedicated server for something like this. Imagine the memory consumption of some applications. A hosting company wouldn't be able to handle something like this easily. Not to mention people who require this type of setup probably need dedicated servers to handle these needs.

PHP is NOT The Next VB6

Thu, 06 Dec 2007 10:26:31 -0700

I saw this article earlier in doing a Google search for dealing with Anemic Domain Objects in PHP. I think the author is really off when it comes to PHP as being related to VB6. His main argument is that PHP's does not enforce OOP style programming the way .NET does (or some others such as Ruby on Rails, Java or #C). He even calls PHP a "Framework." He also mentions that for large scale projects, PHP is not a serious language compared with .NET.

I think these miscues focus on the wrong issue.
His write up targets mostly the issue of PHP not being serious as a language because the object support enforcement is non-existent. He claims that a lot of the internal procedural style libraries disengage PHP programmers from using more OOP programming, which apparently allows for highly scalable, enterprise-class software.

Let's be honest here. He's full of shit.

PHP's problems as a language being ready for the enterprise has little to nothing to do with not enforcing OOP structure. Actually, that's a huge benefit because the ability to learn something like PHP is far quicker than something that enforces OOP concepts like Java. PHP is primarily a functional language, and functional languages have been traditionally easier to teach than OOP languages. My old professor in college once mentioned how difficult it is to teach OOP to new computer science students. Just to create Hello World, you have to teach about OOP theories rather than just allowing something to use:

echo "Hello world";

Not to mention not enforcing the OOP paradigm simply allows people to build software quicker. You don't have to worry about transforming your object into byte classes or anything complicated.

That said, the real problem with PHP has been the fact that most people who are programming PHP do not typically (nor probably) have fundamentals in computer science to create large scale software. Any language can be faulted as being unmaintainable if 1) there is no architect guiding the younger programmers; 2) there is no vision nor design for the piece of software; 3) the people building and maintaining the software are simply "script kiddies."

From a functional perspective, PHP is one of the richest languages around, having numerous libraries that can support many enterprise related task. You have persistent DB connections, XML parsers, XSLT functions, SOAP, WSDL, caching, graphics manipulation, etc. As a language it can scale too as you can create your own cache or use the PEAR caching system or even use horizontal caching with the memcache piece. And you can't say that PHP is not worthy of large scaled projects since companies like Yahoo, Friendster, etc. all implement PHP in their online applications.

Him calling PHP a framework is ludicrous. It's a programming language. A framework would be Zend, PHP Cake, etc. that do enforce to varying degrees (depending on the framework) the notions of good design patterns.

To say that you cannot promote good design patterns with PHP is also ridiculous. I've been my own frameworks based on the outdated MVC models, using numerous design patterns such as business delegates, service layers, MVCs, singletons, factory/builder patterns, iterators, etc. Heck without some of the newer OOP features since PHP 5, I would've had a much more difficult time designing my systems.

I think a better comparison would've been Ruby on Rails to VB6. The thing with PHP is that you can use it as a decent teaching language since it has all the qualities of a decent programming language (you can even enforce types in parameters and do limited pass-by-reference in function calls!). With Ruby on Rails though, you're teaching a limited paradigm, not the fundamentals of a programming language where you can expand the purpose outside much of a web context. VB6 likewise was limited to GUI client and perhaps some server programmer. With VB6, you're limited in the type of paradigm you are programming so the type of thinking that goes into a VB6 application is more bent towards rapid prototyping, just like Ruby on Rails.

PHP provides you with the tools and basics to build enterprise scalable applications. Rails shelters you into a paradigm and makes you think that scaffolding is king, compared to understanding how certain fundamentals like types, function callls, etc. work in a regular programming language.

The other thing is that I just don't buy the excuse that PHP is bad in his context because of not enforcing OOP thinking. And his bad being the equivalent of not being able to build scalable applications. Well, so what would you call all those C and C++ systems out there? C++ does not enforce programmers to program in OOP like Java either. I've seen so-called enterprise level systems for trading securities written in C++, where everything looked more like a csh-script than some elegant piece of reusable code.

As I mentioned before, the language itself does not impose bad thinking. It's that people who adopt a language themselves were never good programmers to begin with and never learned fundamentals of design. I've been on both ends of the spectrum where I saw horribly designed (for lack of a better term) systems built with C++, Java or these heavily praised languages as well as partaking in building monstrously scaling open source systems like Ticketmaster. And I'm certain his much beloved .NET can easily fall prey to poor design in the hands of an incompetent programmer.

I truly wish people stop blaming the languages themselves in the end for poorly designed systems. The only thing languages offer is a slight difference in taste for syntax with a few more features here and there or syntactic sugar coating to make some developer's weener hard because they enjoy the mental masturbation of the puzzle like ways of twisting a language apart.

that was fun!!!!

Mon, 17 Dec 2007 10:02:37 -0700

Ugh, you might've seen some weirdness with the site lately. Well, that's me trying to get some of the new blogging code up. Now, i'm using more search engine friendly files that also might be a bit better in terms of performance from this server. But I'll tell you that I learned a whole bunch from this encounter:

make DAMN sure you use File::close from PEAR if do any file writing or reading. Otherwise, you might end up leaving a ton of files open on the file system. It's just not automatic (sad).
The server, cron and shell have massively different environments that can adversely affect things like paths and environmental variables. Even if your crontab is under the same user as someone who tries to run a script with that login id, you still can get different results.
Using a cache is a real bitch. I'm starting to question using a cache for anything outside of massive websites. After reading an article about how Digg uses highly optimized sql, I started to realize and regret my decision for using a cache system.
Make certain you set your domain in your cookie. I had some issues with sessions without it. It really is a pain since the session gets confused with another cookie getting started, only to be distinguished by the domain.

There's some other things I've learned recently, but I'll write them up later.

guitar hero, electric guitar, PHP script kiddies and Real Programmers

Wed, 02 Jan 2008 06:17:14 -0700

I read an infuriated blog post by this programmer who ripped up PHP programmers for not being "real" computer science people. He also bragged about his guitar ability, saying he could play anything (I doubt it). In the end, I pondered about the arrogance of such a person and brought two things that seemed unrelated together: guitar and computers. Yet the other piece that was missing was this guy's arrogance. A lot of so-called "real computer programmers" hate PHP script kiddies much in the same way that "real guitarists" probably hate the hell out of Guitar Hero.

When I look at Guitar Hero, it reminds me of these scripting languages that seem to devalue "true" computer scientists and their languages (the blogger in question only mentioned objects, closures, which I assume is "real" computer science). I mean, essentially you're making something difficult easier to do for the common person. In the case of guitar hero, you're supposedly making it easier for us to play guitar and be some rock god while being adored by millions of virtual fans.

Or is this the wrong way of looking at the situation?

I think a major difference in scripting languages and Guitar Hero is that scripting languages still allow one to create something. With Guitar Hero, you don't create songs, you barely imitate. And you're not even re-creating something, you're just faking being a star. The one thing about playing a real guitar that anyone who's spent significant time investing their energies into such an endeavor would tell you is that you can actually create art with it as opposed to this virtual toy.

As a guitar player and developer, I can see both worlds. As a guitarist, I realize that I'm not great but I'm still capable of writing small riffs and even having a tiny voice that originates from the way I hit the strings and frets. I've written my own riffs and even put them down on some recording mechanism at one point. I can never do anything like that with Guitar Hero.

But unlike Guitar Hero, scripting languages allow me to create as well. I think it's actually great that we have the capability of things like scripting languages that allowed us to develop so many applications within these past few years. I think scripting languages like Linux has done tremendous work in freeing us from the monopoly of Microsoft. So should so-called "real programmers" unmercifully bash people who earnestly work on things like PHP, but are still able to develop something that's useful to society?

I understand the anger in both areas, but I heavily believe that in scripting languages the anger is slightly misplaced. I think people in development should be proud of being able to create something. If people aren't bashed and are encouraged to improve their skills by continuing to learn how to make better systems, then the criticism is justified. But raw anger and insults don't go a long way.

Now, Guitar Hero on the other hand, is an insult. Guitar Hero makes one a poser because it completely dumbs down the art form of what a guitar is about. If Guitar Hero was able to enhance the ability to create music, it would contribute something to society and culture.

Unit Testing with PHPUnit

Mon, 07 Jan 2008 15:06:55 -0700

I worked a bit with JUnit at my last job, but didn't have as much opportunities to really get it running as I would have liked. This time, I'm working with a lighter unit test system called PHPUnit. It's fairly light and makes coding even more script-like. But what I've been able to do is write small tests for my various method calls rather than attempting to construct a monolithic program and debug each piece one by one.

However, I did encounter two problems, more from a theoretical/approach viewpoint than a technical one. The first problem deals with writing tests for database calls. For example, if you have some complex business CRUD calls that require numbers of data to be in place, how can you insure that it's all setup a head of time? Some people create a backup or test database in isolation for developers while others use something like sqllite or other lightweight databases that allow a developer to further isolate the tests. So in this case, would you write hundreds of test scripts with numerous setup calls for each type of operation, allowing each operation to be run in isolation? Unfortunately, I work alone for the moment but still have a great deal of code to tests, rending the most idealistic points here somewhat futile.

The second problem is dealing with large sets of data during a test. Let's say you want to retrieve data as one of your method calls. Lets say you want to check the validity of the data being returned. So one possible way of handling this might be to load up some external secondary type of data array to compare the results against the method's return values. Again a lot of work.

In both cases, it seems as though they really are symptoms of a "we'll see when we get there" type of problem. Maybe a few tests suffice.

Another non-technical is that I have not gotten into the complete "test-driven" development lifecycle. It's a situation for me where the result set might be too large for me to spend the appropriate time analyzing every situation and converting it into a test case. Or that I'm not exactly sure what I'm really trying to test in the end at times. It seems as though starting with a test is more of a mental state you gotta be in rather than something you can naturally slide into. Perhaps, one major reason why at work (at least) that I have no completely moved in this direction is that I'm re-building something that isn't necessarily from my original design. It's kinda like finding an answer in the dark. While utility classes or functions might work at times for this mindset, business functions are a little harder to conceive at the starting point, at least until I can see the ultimate goal.

Regardless, I can see the value of moving towards unit testing. In some ways, I look at unit testing more like a proof of concept or a science experiment where someone could prove their hypothesis.

PHP Variable Interpolation

Mon, 28 Jan 2008 08:12:22 -0700

As you progress in a language, you'll probably end up getting bored or better yet, attempting to figure out the limits of a language. I'm getting to that point with PHP. Along the way, I wanted to find some neat tricks to either improve the performance of my code, or to make my code more compact, even perl-like (i.e. one liners). I find a neat little trick you can do in a string that's not very well known.

For the longest time, I would concatenate strings together if I wanted to use a method call or remove single quotes from hashes if I'd use them in a string. For instance:

$str = "Hello " . $this->getFirstName() . " " . $this->getLastName();
$out = "My name is $arr[fname] $arr[lname]";

It's said that both have some performance issues. In perl, the second one would be considered decent format, since they often discourage you from quote in a hash. However, in PHP, the second form is considered a bit slower because of some internal conversions that occur with the interpreter. So how can we handle this in a slicker manner? Use curly braces {}.

$str = "Hello {$this->getFirstName()} {$this->getLastName()}";
$out = "My name is {$arr['fname']} {$arr['lname']}";

Both styles are considered good form in PHP, if not bulking up your strings a bit. But you can avoid nasty concatenations or some performance hits by doing this.

Another little trick for handling variable interpolation, especially if you do a lot of inline messages (which isn't a good thing to do, but sometimes can't be helped), is using the heredoc format. Heredoc is a nice technique for avoiding the string concatenation problem. It makes the code appear a little uglier, but you don't take the performance hit of making something more readable by forcing the interpreter to work a little harder. The format goes like this:

$str<< My favorite Japanese famous women are:
1) Reina Miyauchi
2) Norika Fujiwara
3) Shiho
4) Kaori Kawai
5) Ryoko Yonekura
EOL

Perl and other Unix scripting languages provide similar mechanisms. You can put variables and even make method calls in the heredoc as well. In terms of when to use such a device, my recommendation is SQL when you need to create long statements and lack some nice ORM to handle this internally for you. If you have to create email messages or insert HTML into your code, you might as well just skip right over to a templating system like Smarty instead. Of course, if your application is small, then a heredoc wouldn't be considered out of the question either.

Very Productive At Work

Thu, 14 Feb 2008 09:17:17 -0700

Man, you should've seen the amount of code I spat out today. Tons of AJAX/JQuery stuff. Most of it is CRUD with AJAX, but I'm about to build a kind of webtop application that utilizes a great deal of interactive elements with javascript, CSS, and AJAX for handling some administrative tasks. I'm quite happy with the code I've got at work. For instance, my average JQuery function call is around 4 lines while the organization of the code is working out quite well. Stylistically, I've got things done in such a way that my code is EXTREMELY clean cut. Not sure if it's the most optimal, but my application doesn't have extreme performance requirements.

Another aspect I got to work with recently is the Reflection API in PHP. I do some quirky things where I have my authorization piece defined as a combination of the controller name and the action name (where the action part is handled via convention in the Zend Framework). From a search prompt, you can enter the name of an application and it will examine for the controller directories of the application. Then using Reflection, I am able to query the controller for public methods. From there, I reconstruct the authorization permission name and am able to display it. It's quite cool actually because I don't have to manually run all these insert SQL commands to put the permissions into the database.

Also, in the future, I'm hoping to write some web services for Bugzilla, PhpMyAdmin and MediaWiki to create permissions and user accounts from my system. User management is a royal pain in the butt when you have disparate applications containing their own proprietary form of entitlements. I hope to unify these aspects through my administration application and make granting access easier to these applications.

Then my boss and I did a little interesting experiment where we compared the performance of different slurp methods with Perl. We tried using one of the slurp modules as well as this little Perl idiom:

open (<$fh>, "somefile.txt") || die $!;
my $txt = do { local $; <$fh>; };
close <$fh>;

Found out that this piece of code runs REALLY fast. The way he was doing it was building an array using map. However, my theory is that when you're reading a file in line-by-line, it's more memory/IO intensive, compared to storing the whole content into a scalar value. I think in the former case, because you're building a data structure, it's also slower. We tried each scenario with a file around 143 MB. The code above ran at 2 seconds compared to the other routines, which all ran slower.

I swear, it's a totally different world than being at the Anti-Productive company (i.e. HLIKK). I get so much done at work these days. Heck, I have to force myself to leave because my work can be so interesting at times. Considering I've been here only a little over 4 months, I've accomplished a lot and learned a tremendous amount. It makes me wonder how much more I can produce in a year's time?

PHP OpenID Library/JanRain

Sun, 17 Feb 2008 03:36:58 -0700

I'm SLOWLY plowing through this library. It's really horribly documented. There's almost no tutorials and the example code is poorly written. Unfortunately, I haven't seen any other libraries with the flexibility as this one so I decided to implement it.

Well, I've managed to get my code to the point where I can authenticate round trip. Now, it's up to me to finish up the whole login/registration process. Tonight, I hit a major stumbling block where I didn't know that the stupid library uses the $_SESSION global behind the scenes in setting up some key values. In my case, I do something where I shut the session down temporarily. This is for login purposes. However, when I did that, obviously, the library wasn't able to extract the data from the session. Well, isn't that convenient! The sample code for the consumer doesn't mention a damn thing about this. And secondly, I'm now facing an issue where I'll have to figure out if this will disrupt anything during the login process that I wrote up.

On top of that, I spent hours logging various messages trying to figure out where the killer point was. I had to go through those fugly libraries and went a bit nuts placing tons of data dumps all over the place.

Before this incident, I had tried building my own system but had numerous problems trying to get it to authenticate. Then I realized that one of the return points wasn't setup yet and so the authentication server wasn't providing a response. However, the library makes no real mention of this and I couldn't really see for a few days what was going on.

There's, of course, other hitches and hurdles that I'll probably face as I descend lower into the details of the library. But I think when I nail this thing down, I'm going to write up a good tutorial so people won't have to go through the same pain that I'm suffering

WireIt.js and Other Coding Projects

Mon, 24 Mar 2008 10:02:20 -0600

I found a neat little Javascript library called WireIt. With this library, you can create Yahoo Pipes-like wires. But the key underlying technology is actually another Javascript library called excanvas. It's quite convoluted because excanvas is really a wrapper around VML that allows the dynamic shapes being drawn. WireIt also utilizes Yahoo's UI Javascript to handle events, drag and drop and some level of animation. It took some time to get over the initial hump for learning how to handle the thing. Right now, the hardest part is determining how to get the terminal nodes positioned. Once you get it all hooked up though, it's quite neat as you can create visual workflows with Javascript.

Also, I had been working a bit with extjs. I got to utilize the tree aspect. It felt a bit difficult, but again, it's something you gotta get used to. Lots of server programming to structure the output JSON for extjs to correctly format the tree. The library is quite big so there's plenty to learn. I like the fact that the library creates visually appeasing windows and components. My only criticism, outside of it doing the same thing that Swing does, is that it's kinda awkward to program. So far, only jquery has been easy to utilize.

Finally, at home I'm messing around with Amazon's Web Service system. I had long ago signed up for their API key, but never used it as I was too busy. Now, I found some interesting applications and will make use of it. In conjunction with Amazon's Web Service API, I'm using the Zend Framework. I've been quite critical of the majority of it. However, one nice thing is that it has several web service APIs bundled into their system. So today I finally got a chance to dig a little into those areas. It seemed a lot easier than creating my own parsing tool for handling the XML produced from the web service.

Massive Log Files

Sat, 29 Mar 2008 01:30:15 -0600

I had my first opportunity to do some work on extremely large log files. Actually, at work I'm running a batch script that is importing data and I'm quite certain it's still not finished. The thing is that I'm attempting to do some log analysis using Mysql. I managed to put one client's logs, which reached over 4 million rows worth of data for a span of nearly three months. The real test was in my current client where just a few days implied at least a gig's worth of data.

But through this experience, I learned quite a bit from both the Mysql side and PHP. First, PHP really needs a threading model. I cannot emphasize this enough. Most people probably are happy just to do some multi-tasking version of threads, but that's not "true" threads and you really are hacking a model together. I've seen fake threading packages here and there, but nothing that warrants trying it out. I'm starting to see why Java and other languages with threads natively built into them are considered superior in this regard.

Second, global variables are not necessarily an evil. What!?!?!??!?!?! Blasphemer! Actually, using global variables on a few scenarios saved my butt because passing them by value in a function's method calls inflated my memory usage to double the capacity. In those cases, I was loading up massive arrays of information where I needed to re-index some information before loading it into the database. Here's a situation where global variables are actually a GREAT thing because you want to save memory. The key here is that I know which variables are considered global so I can control from a program flow perspective whether or not these variables would have any side effects in the end.

On the other hand, I found objects to be evil. I was using the Zend Framework's models to load part of my data into memory. When you load a rowset using the Zend Framework, you actually are committing yourself to building up VERY fat objects into memory. On small data sets, this doesn't really matter. Most websites on the front end probably don't have to bother with such details. However, when you're loading several hundred rows, it REALLY matters. Fortunately, the rowset object allows you to convert the data into pure arrays. By doing this, I was able to cut memory usage in half. This was a critical step because I had to bump up the memory allocation to near 2gb and the script was going into swap.

From the Mysql side, I'm learned a thing or two about indexing. Most people probably just know about indexes from foreign keys, unique columns or surrogate keys (i.e. sequences). But indexes can be used in more ways to boost up your queries. Here, I was creating an equivalent log file table using with month and day columns. Quite often a person might want to check out the number of unique hits per month, or even for a day. On a data set of say 10 million rows, this can take roughly 20 seconds. Putting an index on day or month can cut the time in half.

One thing I was doing in designing the database was structuring it similar to how dimensional modeling works. Mostly, just one off join tables (I believe these are called the dimensions) from the fact table which was the log table itself. I had several large fact tables requiring me to do up to six table joins at a table. Not deep table joins, but just connecting them to lookup type tables. I tried creating a view to see if I could optimize the select speed. Turns out that the view took longer than a regular query. The reason here is that I would use some filter prior to joining up the tables. The view had no such filter and would grab the entire data set in a joint format before allowing you to run your filters.

Overall, the experience has been very cool because the typical highest number of rows I'd work with might be a few hundred thousand. Here, I'm working with several million, so learning to optimize my code and queries allow me to get a much better feeling for how optimal programming works.

Zend Framework and Web Services

Sat, 05 Apr 2008 13:26:05 -0600

I've been pretty harsh against the Zend Framework in terms as it is far too loosely coupled component based system. However, I've utterly fallen in love with the way the Zend Framework handles Web Services. At the moment, I've embarked on a home project for utilizing Amazon's Web services. If I had to parse the XML by myself, I think I would go crazy. Here's where you really want to have a thoroughly tested mechanism to handle the heavy duty work.

The Zend Framework makes the item search capabilities quite easy. You just need a web service key to be able to search against Amazon's massive item database. The real trick in all of this is figuring out how to tune your searches against Amazon as well as caching the results. Once you get that working, it's quite smooth sailing.

The only problem i see against this really isn't an issue of the Zend Framework, but a problem with Amazon's policy of only allowing a single query per IP address per second. While I understand the reason, I hope one day that they'll eventually lower this requirement. At least the number of queries is no longer monitored, unlike other web services.

php5 with ImageMagick Broken in Ubuntu 8.04 Hardy

Sun, 08 Jun 2008 16:52:58 -0600

My faith in the ubuntu name is quickly falling. This time I found an error with php5 and the ImageMagick extension. Running php on the command line, you'll see the following error message:

PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php5/20060613+lfs/imagick.so' - libWand.so.9: cannot open shared object file: No such file or directory in Unknown on line 0

Apparently, there's a quick and dirty workaround:

ln -s /usr/lib/libMagick.so.10 /usr/lib/libMagick.so.9
ln -s /usr/lib/libWand.so.10 /usr/lib/libWand.so.9

A conversation about this problem is shown in the link.

Nonetheless, this is pretty disturbing. The biggest issue here is that the ubuntu community once again had made a political decision to move ImageMagick off and is no longer in the main repository. Reading the discussion, you'd have several options like using the gutsy repository or even recompiling php5/ImageMagick. All of these solutions are unacceptable to me. The two things that could've easily addressed this situation in the first place are:

Keep ImageMagick 9 in the Hardy repository
More testing!!!!!

I vote on the later. Matter of fact, I think the Ubuntu community ought to perform more tests before releasing these distributions. I can see for minor packages how this type of bug might have little impact. However, I've had stability issues and now major libraries with irresponsible bugs attached. You just can't call a distribution "stable" when things are messed up.

I'm all for progress and pushing the envelope, but this things ought to be balanced with stability in a highly reputable distribution. Having made the switch from Windows to Linux years ago, I made a clear decision to use something that I felt had more stability. Now, I'm seeing all types of faults in the delivery of this system in favor of marketing and attempting to be cutting edge through sticking with a deadline of 6 month periodic releases.

The thing is that for most of us, we just want something that works, is virus free, doesn't crash and stops giving us headaches. At this point, I'm starting to get tempted to pick up a Macintosh, despite my long time prejudice against their hardware. It seems like the last system left that truly is stable.