When I first started the industry of tech, I was indoctrinated by Oracle as my first database and ended up becoming more of a data driven developer as opposed to a requirements driven developer. I think this hurt my overall perspective since my focus for many years was almost exclusively on dealing with 3rd normal form. However, as ORMs, Nosql databases and other non-relational formats and technologies evolved and as the web grew more complex from stagnate pages to more dynamic, real time idioms, I started to realize how my initial views were a huge mistake.
Right now, as I’m learning Ruby on Rails, the thing that strikes me is the notion of how the database ends up becoming a stupid data store. It’s really just an artifact of infrastructure as opposed to the driving force behind a company. In all honesty, I started to become suspicious of relational databases back when I worked for Demand Media and spent time on Livestrong. A lot of problems occurred due to how our Mysql Database would occasionally have issues scaling. My team lead and I started to become extremely frustrated by being tied down through the database layer.
In order to avoid the scaling problem, you end up turning to non-relational methods to handle your problems. For instance, one of our tables at the time was at the half billion row mark. Because of how it was primarily a constant write table, we needed a solution to evolve it. At the time, the solution we ended up going for was archiving most of the data and truncating it. Then when a user wanted to access older parts of their history, we would unarchive it upon request.
But note the language I’m using here. I’m trying to avoid using database specific terminology to describe a problem and create an answer. But that’s where you need to think more on an abstract level rather than a fine grained level such as a database row/table/column at times. And that’s where you need to move from the data layer to a business requirements mind set.
Now, I’m certain that most people in tech will all agree that solutions must be centered around business requirements. Although that’s certainly a common bond, everything ends there since people end up doing work arounds. Quite often, these work arounds exist as a result of the inflexible structure and/or nature of the database. Of course, applications also have their limitations but databases tend to be more rigid because of data.
That’s where I like how systems such as Rails have certain control over more static data elements of data through notions of migrations. Also, things like Rails seem more fluid in design because you’re attempting to do more abstractions. For instance, I encountered the :references value which allows one table to hold a relationship (one-to-many) to another. Rather than bothering with all the details of the database such as foreign keys, indexes, constraints, etc., you allow the application layer to manage that.
Part of the reason why I like this is that the technical details for an application can take away from significant development time. Having the application manage those aspects alleviates developers from useless duties and allows them to focus more on business logic. Anything that saves on time is a good thing for me.
One thing that I have noticed as the industry continues to grow is that the role of the DBA is slowly diminishing. I feel that DBA came about as a result of Oracle being so dominant early on and existing solutions not having an open source, well documented alternative. In turn, DBAs ended up becoming extremely powerful, commanding high salaries and often times enacted fairly draconian policies with regards to data. More than that, they would attempt to control the business layer by forcing developers to house all their business logic within the database. In turn, that forced applications to utilize incompatible SQL and stored procedures and providing little flexibility as new cheaper and potentially faster/better solutions appeared on the market.
Unfortunately, you had varying qualities of DBAs. Some were actual computer science students who knew about application development. However, others seemed more like people who just had fancy certifications but knew and potentially cared little for application development. Instead, some of those more poisonous individuals would utilize their power as DBAs to have all decision making processes filter through them. If they lacked the passion and/or knowledge of application development and also were difficult people (and quite a few that I’ve encountered were), the application development process would end up becoming a huge, unnecessarily bureaucratic nightmare.
The thing is that as a developer I want to build things without the issues of unnecessary forms, reviews, etc. to get to an end solution. As an architect, I want to develop solutions without thinking about artifacts upfront. I want to focus on the problems then break down the problems rather than starting with say a table and defining all the columns. Once you go down the latter path, it becomes a slippery slope because you end up working around the table as opposed to the problem and making that solution more scalable.
That isn’t to say that data isn’t important. Of course, these days it becomes a huge part of the genetics that compose an organization. But in the application development process, I really do not like it being the center of attention. Similarly, I think picking a technology in advance and trying to fit it with a business problem is equally wrong. Focus on the problem first.
Lastly, I talk a lot about Ruby on Rails. Although I decided to pick Rails for my next project, it’s mostly because I want to learn about it and how to really utilize it. That is somewhat different than say arbitrarily choosing Nosql as my data store because it’s the latest and greatest. If it’s a learning situation, then choosing the technology in advance is fine. But don’t play with others’ money if you have the chance.