Is it possible to do manual unit testing?

Unit testing by definition is testing of individual units where a unit is the smallest testable part of an application for example a class method in isolation. What it means is, we have to ideally test just the logic within the method and not the logic of supporting classes or libraries being used in the current method.

Now we can do this through an automated tool like junit/testNG/nunit/flexunit etc by creating stubs and even more often now mock objects which are not real objects but still mimic their behavior making it possible to unit-test functionality of a single class without actually calling underlying or collaborating classes. How is it possible manually?

The answer is NO, it is not and thus any form of manual unit testing is actually integration testing.

Sharding with MangoDB

As per my earlier post we need to scale out our data store when the application data requirements outgrow a single instance capacity. This is in most cases achieved through sharding (a.k.a. partition of data).

With traditional RDBMS systems the application data model has to be modeled such from the very start to enable equalized partitioning of data or it may require a considerable change if we want to do it as an afterthought to scale up the application for using multiple database instances where each instance keep a balanced subset of the big data for an application. So in a way we have to plan for scaling out an application in typical RDBMS world but then with MongoDB it can actually occur as an afterthought as it provides automatic balancing and fail over of the partioned data

MongoDB as on 1.6 onwards supports production ready automated sharding architecture where we can convert a single instance into a cluster of shard whereby MongoDB manages the distribution of load of data across the shards (redistributing if the data on one node goes out of proportion with the rest) and also provides fail over support with each shard data being replicated in a replica set and all this with very minimal or no code change. The application needs to connect to a mongos process (which sits in front of the sharded cluster) which is responsible for managing the reads and writes across these shards giving a similar impression as we are working with a single node database systems.

Web Applications Scalability

Please do take care of the scalability concerns of your application or at least architect it such that it can be scalable later on.

If we ignore this fact at the very start concentrating more on the business aspects of the application (something which is misunderstood in Agile) we are doomed to fail even at the slightest hint of making the application scalable. Do not get me wrong, there are measures which can be taken a little later in the development cycle which can improve the scalability of the application but then they have their limitations. For example something like what to choose as data storage is something you need to decide upfront. Agreed having designed the application with a layer of data access objects abstracts the rest of the application from the data storage dependency but then why do the rework of rewriting the data access implementation. I am very sure we cannot regard such a change as trivial.

Have a strategy to allow the application to scale from the very start will definitely help and it is more a matter of choosing the right technologies and right architecture to accomplish it upfront.

TDD and Architecture

Let us first look at the TDD and why I feel it is right way to develop functionality. To start with let me just ignore TDD as if it does not exist but obviously I do believe writing unit test cases is worth the effort.

So I have done my design, written my implementation as per the design and now I am writing the unit test cases to test out every possible scenario and then I realize I am not able to unit test a scenario until I modify the code to get it more suitable and enabled for testing. I modify my code to make it available for testing but then we know if our code is not testable, it is a flaw in the design and the design itself should have been such to allow for it to be tested. So I need to modify the design.

So in a way my tests are driving the way the code is written and eventually the design hence why not start with the unit tests at the very first place rather then going through an unnecessary loop and that is what is TDD (or more aptly TDDAD – Test driven development and design).

But then can we just stretch it even further and have my test cases drive the architecture of the system as well. Not really

1. Architecture by definition has much wider scope and relevance then may be just the design of few classes. It is a decision which is taken at a team and solution level and not merely confined to the reasoning being done at the pair programming level.

2. We need to get some basic architecture in place as part of zero iteration which takes in account just enough skeleton in place from which we can start building. We need to ensure we are pretty confident of this basic skeleton as it will be one of the hardest stuff to alter later in the future iterations and will have quite a strong consequence whenever we plan to do it.

3. Architecture decisions are very hard to postpone as something like going for a typical n-tier architecture or may be for SOA or in some case opting for a third party off the shelf solution or the very latest GAE as PaaS solution has to be taken upfront and cannot be delayed for a point in future

Not to say that we need to do big upfront architecture but still the bare bones or the minimal set has to be in place and then let the design evolve

Management as Craft

Management in general is very generic and might involve very varied role and responsibilities depending a lot on where we are. Not too kill the fun of hundreds and thousands of job titles floating around at the moment within IT with the suffix as Manager, I dare to list down some of the very basics one must possess in my opinion but then it can never be comprehensive enough to cover most of them.

1. Earn their trust and trust the ones you are managing as a team lead – Give them the space, enable them to be effective, keep them out of the politics and unnecessary management value and vision dialogues, do not over inform them of the goals and objectives as they very well know the importance of the business and how much impact they are already doing through high quality deliverable. Make them comfortable and protect them to gain their trust.

2. Remain connected always but not micromanage – Make sure they feel valued and even more their work by asking smart, relevant queries even if you the answer to many. Never micromanage and leave them to manage themselves just being sure they are very clear on the goal and timelines. Development and coding background especially ability to help with the design and architecture decisions really help and help you being closer to the so called technical crowd of the project

3. Let them own and feel responsible for their work – Empower them and let them handle it giving right support and guidance throughout. Do not make them agree to you all the time but challenge them asking for ways to improve and optimize the work and velocity

4. Do your bit for their morale – Nothing is good for the team as much as highly motivated and energetic team working with you. Inspire them , reason it out and keep them informed about all the important decisions no matter of it effects them directly or indirectly. Not just 1/10th but atleast 8/10th picture of the importance organization changes happening and why

5. Be there to say “No” for the team – Never leave it for the team to come back and say no to you. Be the first to take the shot if you really believe if it may cause distraction to the team to meet their goal and commitments, the team will not be able to complete it, we are not going to sacrifice on the quality, the testing team doesn’t have enough to get started with the testing.

I know there are a lot more and may be I will update the post as I evolve more as a manager and learn this craft which actually doesn’t have that many hard ground principles or rules as such

Leadership Smells

Just like code smells, I just wanted to highlight some irritating and disgusting practices people follow in the name of leadership. Do these if you want to join the same group of …………

1. Behave, act and communicate what you are doing as path breaking, out of the world, big value add for the organization
2. Your sole aim of existence is to impress your boss, be on his side explaining your achievements and how much support you have got from him/her to achieve them
3. Always lunch with you boss and never with your team.
4. Try to keep things confidential as much as possible to enjoy being in the elite group
5. Keep the time which you need to spend with your team as low priority over everything else
6. Reorganize everything as per your convenience once you move into the new role to demonstrate your value and the impact of your role and hierarchy
7. Delegate the responsibility not the credit or recognition of the good work done
8. You be the single point of failure for any ideas or suggestions coming from the team you are leading
9. Never challenge your boss or even you have to, to demonstrate such a behavior do it where it impacts the least
10. Have few favorites within the team. Trust them blindly, they are always right

First Stop to Cassandra

So I start my search for a non relational database one more time to see if the latest buzz about NoSQL really worth it or is it just another time when we are looking at the options which eventually are going to die down for the ever dominant RDBMS.

I soon realized NoSQL encompasses much more then I would liked (just because I find myself a little lost with so many options) with so much already out there challenging the SQL databases. My first stop Cassandra.

Why Cassandra? – For me at least to start with the need for a non RDBMS has been quite straight forward, a data store which can provide me high availability with complete fault tolerance, highly scalable, very minimal downtime if any. This is mostly achieved through highly replicated and db clustered environment. Cassandra comes out as a natural choice to explore.