The Myth of Job Stability

If you feel comfortable, stable, and nothing seems to spoil your satisfaction with your current job… you should be alarmed.

It means you have found yourself a comfort zone. And while comfort zones themselves are not bad, what’s bad is staying there for longer than you should. Whenever you settle into an organization, and before you bask in your new job’s comfort, you should consider or at least have an idea about your exit plan: when, and to where.

Job Stability? Oh, sure.

There’s no such a notion as “job stability”. Stability goes hand in hand with your competence. As long as you’re competent, needed, and its impractical to replace you, you have stability. And only for some time.

Realizing this fact is the first and strongest motive for planning and strategizing ahead for your career. The moment you land the job you always wanted, preparing for your exit from that dream job might well be your best insurance policy.

But I Want The Corporate Ladder

Some people have a certain perspective. They think, I want to “stabilize” myself into that organization, and advance my career inside. Fine, that’s a typical and respectable point of view. However, you should have well-thought answers for the following questions if that’s the case:

• What is that you really want from your career?
• Why define your career in terms of your organization’s career ladder rather than your own vision?
• What can I deliver more than my peers can deliver anyway? What are my differentiators?
• How long could it take me to achieve my targets inside the organization?
• How long could it take me to achieve my targets somewhere else?

That’s not an exhaustive list, but the purpose is clear. You must always keep an eye on yourself, your status within the organization, and above all must always seek to increase your competitiveness. Never get complacent and uncritical of yourself, and cut the feedback cycle because of illusive stability. Thinking that way is healthy, for both you and your organization.

“Take control of your destiny or someone else will” – Jack Welch

QuickFix – (DB2 UDB) Resolving the SQL0444N – Reason Code 9 Error

I’ve encountered this issue lately with a DB2 Net Search Extender (a legacy search engine Add-on for DB2 that enables free text search) implementation which has been running smooth for over a year in production until this error showed up .

Quite simply, its an out of memory condition that may happen when your program invokes a Stored Procedure that has unusual memory requirements for its input parameters or its output.

The resolution is simply to avail more heap memory to DB2 Agent Processes. This is done by increasing a DB2 configuration parameter called the Application Support Layer Heap (aslheapsz).

Execute the following steps to resolve the issue:

1. Open a DB2 “Command Window”

2. Execute  the following command. The number here is completely up to your application’s needs. Each unit is 4KB, so using 500 as a value would avail 2000 KB of Heap Memory for Agent Processes.

db2 update dbm cfg using aslheapsz 500

3. Restart DB2 for the change to take effect, and you’re done.

Performance Engineering of Web-based Applications: 4 Essential Guidelines

When one puts “high performance”, “scalable” and “web applications” together, what comes to mind are websites of the likes of Facebook; sites with millions of users initiating requests of all kinds every single second to what seems to be a black-belt ninja system which, against all odds and adversaries, manages to respond flawlessly every single time… but ahem.. let’s not get carried away.

I’d envy you if you’ve worked on projects of that scale. But, if you’re like me, the scale of projects you’ve worked on has probably ranged from few users to few hundreds or thousands at most. Well, the good news is that there’s a lot of exciting principles to learn when attempting to engineer a system for performance, and they apply to both small-scale systems and large ones.

The Three Basic Measures of System Performance

The performance of a given system when processing a certain Workload can be measured in terms of three major characteristics:

Resource Utilization – The amount of system resources utilized to process a Workload including CPU, Memory, I/O, and Storage. Normally, the objective is to keep Resource Utilization measures in preferred “normal” ranges when the system is processing the Typical Workload.

Throughput – Simply, it is a measure of how fast your system can process Workload Units, regardless how you define what a Workload Unit is for your system. “How fast” is measured in terms of number of Workload Units per unit time, usually second or millisecond. Your Workload Units might be anything from database transactions to file reads/writes.

Response Time – Probably the best indicative performance measure of all – for a single obvious reason – it directly affects your End Users’ experience. It is through Response Time your End Users perceive the overall performance of your system. That being said,  it usually makes sense to gear your Performance Engineering activities towards improving this measure first.

All performance engineering activities aim to improve upon these measures one way or another.

Sketch Your System’s Workloads

- What is a Workload?

I’ve mentioned the term “Workload” many times already and because this concept is so central to the discussion of Performance Engineering, it has to be defined no later than now. A Workload simply is an amount of Work your system is expected to carry out during its operation, defined in terms of specific measurable units such as Transactions. When I mention the “Typical Workload” (or TWL), I mean the Workload under normal operating conditions with the number of users and resource utilization planned for initially during the inception of the system.

However, what are far more interesting to us are the “Peak Workload” (PWL) and the “Spike Workload” (SWL) respectively. Since the Workload of your system is likely to vary over time, it is normal that there will be peaks and troughs. The Peak Workload represents the highest Workload you’d expect a system to encounter and be able to handle at some point during operation. The Peak Workload occurs during the “rush hour” of system use, and may last for hours or days.

The Spike Workload is an unusual, unexpected, and short-term surge in the Workload imposed on your system. For example, the media may suddenly volunteer to provide coverage  for your shiny website causing an unexpectedly large number of people to rush to check it out, causing a Spike Workload.

A vivid example of the damage caused by spikes is when fark.com or slashdot decides that some website has a good piece of news worthy of its readers’ attention. As soon as a link to the featured website is posted, bazillions of users overflow the poor unsuspecting website, all in a very short duration. HTTP Error 500.

The all important question here is how your system should react when it encounters such kinds of Workloads. The answer: your system should be designed  to handle the Typical Workload with ease. It should be able to handle the Peak Workload adequately as well. And since the Spike Workload is by definition unexpected, your system should respond to such condition by 1) alerting system administrators, and more importantly: 2) degrading gracefully.

Graceful degradation could be considered a part of a greater concept called Homeostasis, the ability of a system to regulate itself to maintain a stable condition. Graceful degradation has to be taken into consideration during development, especially when the system has to be highly available.

Performance Engineering: A Set of Guidelines

Now the following guidelines are just plain common sense. These guidelines should best be followed during the development lifecycle to identify which areas of the system need to be designed with performance in mind. However, you might be assigned to tune an underperforming system, and in such case, you should also start by these guidelines — but armed instead with usage data gathered from a real-life production environment.

1. Set Your Priorities Using the 80/20 Rule

First, you have to identify which functions (or Use Cases) in your system need to be engineered for performance. It makes sense to start by focusing on the 20% of Use Cases that are done by end users 80% of the time and create measurable Workload. If you have 1000 users who would have to perform a “Search” use case 80% of the time for example, that makes the “Search” use case a good candidate for your focus.

Also, if users of your system are divided into groups (by role for example), then you have to do the previous analysis for every user group.

2. Set Your Performance Targets

Having identified Use Cases that are good representatives of  the Workload, you need to set their associated performance targets in terms of our three essential measures: Response Time, Throughput, and Resource Utilization. These performance targets have to be set with customer stakeholders.

You start your discussion with your customer by the performance targets of the business itself.  If the business requires that a user be able to complete 600 transactions per day, there you go. By the end of the discussion you should have enough information that can be easily translated into system performance targets in terms of Response Time, Throughput, and Resource Utilization. Sometimes that translation is easy, sometimes you’ll have to make many assumptions. While its OK and even necessary at times to make assumptions, they have to be documented and justified.

You also have to consider with your customer when the system is going be most heavily used daily, monthly, or yearly. For example, in the banking sector, usually the end of every month is a pretty busy time. Also, how many concurrent users should be supported by your system for every user group and therefore, as a whole?

This discussion will ultimately lead you to what are the expected Typical and Peak Workloads for your system. But that’s not the whole story. What about future growth? If the business depending on your system grows, so would your user base. Growth is even trickier with public websites, because you cannot know the extent and duration of this unbounded growth.

Following is a simple example of sketching Workloads for a hypothetical Item Ordering System, just to demonstrate what I’ve just discussed:

[Typical Workload]

3 Administrators concurrent, workload-representative Use Cases(Monitor Orders 2000/day)

300-500 Users concurrent, workload-representative Use Cases( Search 5,000/day, Browse Item 60,000/day, Order Item 1000/day)

Response Time: 2 Seconds maximum

[Peak Workload]

4 Administrators concurrent, Workload-representative Use Cases(Monitor Orders 3000/day)

1000-1200 Users concurrent, Workload-representative Use Cases( Search 10,000/day, Browse Item 120,000/day, Order Item 2000/day)

Response Time: 4 Seconds maximum

3. Analyze Candidate Use Cases

So, you have identified Use Cases that represent Typical and Peak workloads for the different user groups. You know which Use Cases need to be engineered for performance and scalability. What’s next? You’ve probably guessed it, delving into the design and implementation details of such Use Cases. The higher and more stressful the Peak Workload is expected to be, the more attention you’ll have to give to the details. At the extreme, you may have to aggressively tune algorithms and optimize memory usage.

4. Prepare Your Tools

Finally, you need two kinds of tools when doing Performance Engineering: a Profiler and a Performance Testing tool. The Profiler is your informant that gives you valuable insider information – and sometimes even amazing insights – about your application’s run-time behavior including Memory footprint, execution times of Class methods, and even code coverage reports. However, these features are highly tool-dependent.

As for Performance Testing tools, they are invaluable for helping you test the performance of your system under different Workloads. JMeter is one open-source performance testing tool that is pretty stable now and has been around for a while. Also, I recommend IBM’s Rational Performance Tester if your organization is willing to invest some cash.

Performance Testing tool makers are aware how challenging it is to design, implement, and most of all: execute Performance tests. That’s why you’ll find some advanced features common in most tools; such as proxy server-based recording of HTTP activity for example, that allow you to record test cases and then replay them against your system in volumes.

Hey! What About Scalability!

Last, a concluding remark. A concept that is closely related to Performance is Scalability. Scalability is literally the ability of a system to “scale” with the increase in workload.

I had to mention Scalability because there’s a widely spread misconception about it. Contrary to a very common belief, Scalability is not a system’s ability to handle a heavy workload. That should be called “a properly engineered system” instead.

Scalability is actually all about how easily an entire system – both hardware and software – can grow (e.g. by adding more hardware or replacing aging machines by more powerful ones) or be modified (by virtue of flexibility of application architecture and design) to handle Workload increase beyond the point it was originally engineered to handle. Especially if more functionality is added to the system over time as part of the system’s evolution.

That’s all for now, folks!

Pragmatic Program Design

You are asked to create the architecture or design of a program/subsystem/component. What is the most practical, timely, value-adding approach to do it? While this article doesn’t definitively answer this question, it attempts to get as close as possible to doing so.

Any program design effort is primarily concerned with organizing program logic (in the form of code) into what can be called “program units” such as functions, methods, or classes; and on a higher level, components and subsystems. But not only that. Design is also concerned with making implementing and changing program logic an easy job that can be done efficiently.  That means with minimum effort and in minimum time by the developer.

The beauty of defining program design as the act of organizing program logic is its simplicity and fundamentalism. Think about it. When you decide, for example, to put a specific piece of code in a class method, you’ve made a decision about organizing a piece of program logic. You’ve placed it somewhere and given it a name.

Now, with that notion in mind, how can you differentiate a good design from a bad one?

The Code is the Ultimate Judge

For me, the Code is what has the final say on the quality of a design. Design exists as an activity to make code readable… understandable… reusable… and most importantly, to make code easily accomodate foreseen change. If the code exhibits these traits, then it can only be the result of good deliberate design.

Writing code that has these good traits depends on many things such as the support of the programming language itself for code organization, the complexity of the problem that is being solved by the program, the foreseen changes that are possible or likely to happen in the program, and the developer’s lust for neatly written code.

As an example on programming language support, Object Oriented Design was a remarkable step towards enabling good program design. OOD came to the rescue after Procedure Oriented Design failed epically with the increase in code size and complexity. OOD provided ideas for better organization of program logic. Instead of functions, procedures, there are classes, methods, and packages.

All OOD’s foundational pillars: Encapsulation, Inheritance, Polymorphism, were geared toward helping making code more understandable, reusable, and elegantly capable of accomodating change… in other words, OOD enabled better designs by promoting better code organization.

Pragmatic Program Design (PPD)

Since program logic is materialized in the form of code, PPD is a bottom-up approach that essentially starts with writing code, not drawing diagrams. I claim that to be able to elegantly design a program, you have to implement it first. This might seem like a silly riddle, but let me show you why this approach has far more common sense into it than the usual “model first” approach.

Traditional approaches encourage addressing all aspects of the program in the initial design effort, perhaps even from different views (usually a variation on the 4+1 Architectural Views). This is evident in a number of software processes such as the Rational Unified Process and others.

You start by component modelling: creating a set of components and distributing program responsibilities among them. You first “somehow” understand what the responsibilities are, and then create components to carry them out. You end up with a document or a diagram that says Component A does X and Component B does Y.

In order to model components, 99% of the time designers would just use their imagination and prior experience to gain an initial understanding of the different program responsibilities. They imagine the steps that will be taken by the program to fulfill a specific required Use Case. Designers start by the important Use Cases and imagine the happiest possible scenarios: the ones where everything goes right.

Armed with fresh understanding, designers rush to their modeling tool to diagram components and their relationships and interactions.

Big mistake.

The biggest fault done here is abstracting too early. Abstractions as high-level as architectural components are supposed to be long-lasting and stable. They must not be based on early immature understanding of program responsibilities. The resulting component model in such case is always naive, and is highly likely to change very shortly afterward as a result.

Instead, the best way to better comprehend and appreciate a program’s responsibilities is to implement the program. Implement it as quickly and simply as possible. The focus here should not be on writing clean organized code. It should be to gain insight into program responsibilities and how they interact. You can always refactor and improve the organization of your code later when your understanding allows you to do so with ease.

After you write some code, and get some sample output, and feel that you’ve covered several happy scenarios and reasonable special cases, only then you start thinking about how you organize code into components. At that point, you are armed not just by theoretical understanding, but also by insight given to you by the quick implementation you have coded.

Actual design starts in PPD by re-distributing responsibilities on classes, creating the classes on the way. Immediately when you look at the code you’ve written, you’ll be able to spot parts of the code that are good candidates for abstraction. The best thing about starting your design bottom-up from detailed code is that you are more immune to over-engineering your program. You will only abstract and generalize when it makes sense to do so.

Now, which sounds to you like a better design approach?

To conclude this section, here is Pragmatic Program Design in three simple steps:

Write focused straightforward code FIRST,

Make it work,

Then organize and refactor code into a suitable design.

PPD is nothing but pure common sense. In fact, I’m sure that many people out there are doing it already. I’ve chosen to write about it here, however, in order to help rising designers accelerate their learning curve and avoid common beginner pitfalls.

However, there are cases when doing upfront design modeling can actually make sense. When? It is when you are so familiar with the kind of program at hand and have written it many times before to the extent that you know by heart the responsibilities involved and the best way to organize and distribute them. Its when you can strongly assert that the design will not change substantially later on.

PPD is Great for Validating Designs

The Pragmatic Program Design approach can be applied selectively to devise good program designs on all abstraction levels from architecture and down to detailed class design. PPD provides high value especially when you’re designing at a high abstraction level, because that’s where the detailed information needed for producing practical stable designs is missing. PPD helps you fill-in the missing pieces by promoting writing focused code prior to elaborate design.

For example, let’s say you’re thinking about using the “Layering” architectural pattern in your program. You want to validate whether the layers you’ve come up with are practical or not. A good way to get your answer quickly is to do PPD. Write code that contains the logic from all the layers, and see whether you can divide that code somehow into your supposed layer structure.

After all, layering may turn out to be a bad idea in the case of your program’s architecture, but you’ll never know that for sure until you write code.

Concluding Remark

I hope I have detered you from being the perfectionist “Designer” who gets carried away spending days and nights “designing” programs… only to end up with many neat and fancy Visio diagrams that represent your best theoretical thought without adding much value.

In the next article, I’ll discuss how to best incorporate Design Patterns in your designs when applying PPD.

Few Observations on Software Development in Egypt

While I’m not an “authority” on the state of software development industry in Egypt; I’ve been in the market for 5 years now, closely observing the surroundings. I’ve come across 5 different software development houses with significantly different size, culture, and overall competence.

On this basis, I’d be happy to share with the community some observations on software development in Egypt.

Feel free to provide me with your feedback and thoughts if you believe you see things from a different angle. I’m very much looking forward to having a thoughtful discussion with you.

Market Growth

The Egyptian software market is growing, thanks to the relatively low cost of human resources and the initiatives recently adopted by the government. The sudden boost given by the government to the industry (more on that later) created a sharply high demand for qualified calibers; a demand that is still severely unmet at the moment of writing this article.

Note that I don’t have concrete numbers to support this observation. However, HR strategies in the places I’ve worked for in a period of 5 years clearly support this observation. Also, the government has lately launched some ambitious initiatives with global giants such as IBM, creating more and more IT vacancies now, and in the future.

That’s why I speculate that a professional IT-focused training business would be a huge success in Egypt. There’s a clear unmet common need that software development houses could not fulfill alone. Many software houses currently resort to in-house training, usually with suboptimal results.

So, if you’re thinking of opening an IT related business here in Egypt, training should be among your first alternatives. But if that’s so easy and obvious, why haven’t someone already done it? I have no idea. Lack of qualified trainers might be a reason.

Human Resources and Hiring

The Human Resources function in a software house is one of the most impacting on its overall performance. Yet, few companies realize that. 

The Shotgun Approach to Hiring

In a booming market such as the Egyptian, financially-capable organizations often resort to the “Hire now, think later” philosophy to their own peril. They adopt the “sinkhole” or “shotgun” methodology where massive hiring is done to rapidly achieve number-based targets. I’ve seen two companies doing it.. with dramatic outcomes.

The greatest problem with that mode of hiring is that quality of candidates is often severely compromised for the sake of achieving number-based targets. This problem is even more serious if the HR function itself is weak, with the primary weakness being the lack of methodology in judging candidates’ competence.

The consequences? Every incompetent candidate brought in lowers the overall competence of their teams and eventually the whole company. With enough incompetent employees, an organization would rapidly turn into a no man’s land for the bright people.

Another problem is that few of such companies can adequately handle rapid growth. If not prepared for, rapid growth immediately leads to serious management and logistics issues.

Software houses should shift to hiring for quality before quantity.

What if high quality candidates simply aren’t available or affordable (as in the case of Startups) ? Then they should go for candidates who are self-motivated (usually they love their job and seek technical challenges), fast learners and have superior soft skills. There is a variety of ways to test the “fast learner” skill/talent. My favorite is to present the candidate a reasonably complex multiple-day assignment that may require the candidate to learn a technology to get it done.

Need to Hire and Retain Superstars in Egypt? Here’s how.

Hiring superstars is done through contacts. Superstars usually won’t apply for jobs in papers. They’re already safe, secure, and satisfied in their jobs. Google is the smartest company that approached hiring here in Egypt. Just like the GMail invitations scheme, Google only interviewed candidates who were recommended by other competent candidates. After all, who would recommend a sloppy candidate for Google? That’s why when Google came to Egypt, they’ve reached the top superstars much faster and cost-effectively than most other companies using the traditional ad-based approach. 

That’s why when a software house comes to building its HR function, it has to hand-pick recruiters who either already have wide networks of contacts, or those who know how to build them quickly, and who are proficient using social networking tools such as Facebook or LinkedIn.

But there’s another set of interesting questions. What motivates the superstars here in Egypt? How can you keep them once you hire them? How can you attract them if you’re a start-up and cannot afford their paycheck?

There are fairly obvious common sense answers such as “Higher Salary” and “Interesting Technical Challenges” and so forth. The real question here is: which is more important for an Egyptian star performer?

My experience says that “Appreciation” and “Accelerated Career Path” top the list here in Egypt.

The most prominent reason is that we Egyptians are emotional people. In our culture, the saying “A little flattery gets you everywhere” can’t be more true. 

Actually, many star performers may even be happy with an average salary if they feel they’re highly appreciated by management, and believe there’s a chance for rapidly climbing up the corporate ladder either in a technical or managerial direction based on their preference.

Smart software houses should work on their career ladders in order to allow rapid growth, and to allow easy transition from the technical path to the managerial path.

The “height” of the career ladder isn’t as important as the length of time it takes to move from one step to the next. It might be a long way to reach managerial level, but that doesn’t really matter if there’s a feeling of acceptable progress along the way. Progression can make the impatient patient.

Popular Software Development Technologies

Again, I have nothing but empirical observation to support the following claims. All the following observations are related to Information Systems and Website development.

Microsoft technologies are hugely popular in Egypt. Although Java has been lately gaining popularity with the support given by giants such as IBM, dotNET platform technologies prevail. 

The dotNET platform is adopted by the masses mainly due to its relatively shallow learning curve and the high productivity it enables.. and, because simply there’s no much of choice about it. Microsoft provides the developer the Visual Studio IDE and it runs on Windows, period. Microsoft also pampers software developers by giving them an IDE that just works with all the bells and whistles.

In Java? There’s a myriad of choices about the IDE, the Operating System, and the supporting technologies to use (version control, build system, etc.).

Bottom line: it is far less easier and faster to become a professional and productive developer with Microsoft technologies. This is good news for some and bad news for others. Its good news for Java developers, because that means there’s fewer of them and they can earn higher salaries. Its bad news for start-ups who base their business on Java for the same reason.

Lack of Java-focused professional training services is another important reason, in my opinion, for its lack of popularity among developers in Egypt. Sun Microsystems also seems to care far less than Microsoft for spreading the adoption of their products in Egypt.

That’s why in Egypt, dotNET wins over Java in terms of popularity.

Its noteworthy to mention that the WAMP (Windows, Apache, MySQL, PHP) stack is also immensely popular in small companies that live off building small-scale websites. PHP developers are many and they’re well in business.

There’s a lot more to say on that topic, but for the sake of brevity, I’ll stop right here and move on to the next observation.

Management

The Egyptian software industry is still on its path to maturity. Currently, old-school Management is still dominant.

It is rare to find management that seriously adopts new software development methodologies such as the agile family of methodologies (XP, Scrum). Note how I emphasized the “seriously” part. There might be adoption, but on the slightest hint of schedule pressure, usually methodologies are thrown out of the window. Only random application of few of the practices promoted by such methodologies remain. Waterfall is still alive and kicking everywhere here because both management and customers are comfortable with its linearity.

College Education

Throughout my career, I’ve seen a number of rising Superstars. Superstars that are “On Fire” kind of productive, those that I could blindly rely upon to get things done in time while trusting that I’ll eventually receive high quality deliverables. When asking about their backgrounds, I intentionally ask about their college education: where did they take it?

The answer was usually one of the following (in no specific order):

The American University of Cairo (Computer Science Department), Cairo University (Faculty of Computer Engineering and Faculty of Computer Science), Ain Shams University (Faculty of Computer Engineering and Faculty of Computer Science), Helwan University (Faculty of Computer Science).

That of course doesn’t mean that great calibers graduate exclusively from those schools (or even have relevant college education for that matter). It just means that when a company hires a graduate, there’s a high probability he or she will perform relatively better.

These schools are better than others because they teach what is more important than CS curricula: they teach hard work, commitment to results, and being fast-learners. At least two of them place good emphasis on creating a graduate who is clever not only in science, but also in interpersonal skills.

However, I can safely assert that IT College Education in Egypt in general is detached from the real world, and is not sufficient to launch a successful career. Only recently some of the top universities are playing catch-up with the blazing fast IT world.

Except for the top 4 or so universities, many companies make the big but understandable mistake of blindly trusting candidates holding the famous 9-month ITI Diploma over college graduates.

Bottom line: software development houses should take into consideration college education when hiring, and be prepared anyway for heavily investing in training.

Industry Support By Government and Community

Government

The government seems to be dedicated to pushing the industry forward. This is evident in the many strategic initiatives signed by the government with leaders such as IBM and Microsoft. Egypt has become now a really promising place for IT business as a result of this dedication by the government.

But that doesn’t mean there weren’t mistakes.

Facing the fact that qualified calibers are rare, government decided to launch yet another mass-production initiative to help satisfy the peaking demand for software developers: a 9-month crash-course program designed to “qualify” the unqualified. The program is granted by the government free of charge for those who would pass the entry tests. This program is famously and casually known as the “ITI Certificate”.

While the government should be thanked for their well-intentioned efforts, there were unhealthy consequences that affected the market as a result of a fundamental truth: a 9-month program can never produce the same value as a 4-year university degree.

Add to that that there’s  no way you can know how an ITI Ceritifcate holder performed in the period of their 9-month program. In fact, you can never know whether the holder is a lousy or star performer. The only guarantee you get is that the certificate holder “attended” the 9-month program. How reassuring.

But over and above everything else, a 4-year degree gives what a 9-month program cannot: the chance for an individual to realize and practice the real “secret” to being successful in IT and software development: being able to learn and comprehend new technologies and be productive with them in a very short time-frame. The ITI program in contrast is focused exclusively on providing the minimum education and experience required to work with a specific technology. 

There’s no problem with creating a 9-month program to satisfy immediate market needs. However, this should be regarded only as a temporary solution until the real problems are addressed: (1) the number of IT schools and their capacities must be increased and (2) the provided education have to be significantly “tuned” to produce graduates who can be immediately productive and meet market challenges.

Community

As for the community, there is practically none. Only feeble but respectable attempts are made here and there by motivated and ambitious individuals (e.g. EGJUG), but there’s no adequate sponsorship that is necessary for a thriving community neither from governmental entities (ITIDA, public universities, etc.) nor commercial entities (Companies, private universities).

It is worth to mention that there’s some governmental entity called the Software Engineering Competency Center but I’ve heard very little about them, and they don’t seem much active to me.

Recently though, Microsoft launched an initiative called “Hero” that aims to be the “Egyptian IT Professionals” community, hosting activities such as free monthly sessions.

Let’s hope Microsoft’s Hero initiative doesn’t turn out to be merely just another evangelism podium.

Team Sizes that Best Work in the Egyptian Culture

One.

Yes, you read it right. A fact about the Egyptian culture is that most Egyptians value individual over team. This is evident everywhere in the Egyptian society. Unless within teams of people mature enough to overcome the cultural norm, everyone’s comfortable (management and team) when there’s the hero and the star. If you must have a team of more than one star performer for any reason, clearly divide territories of responsibility, or else nasty things might happen… especially when inflated egos are involved.

On a separate note, Team Building 101 says that productive teams that have chemistry among their members are hard to assemble and take long time to build. A team passes through four commonly known stages to finally become productive: Forming, Storming, Norming, and Performing.

Based on my experience so far… it is way better that you short-circuit the “Storming” and “Norming” stages. How?

The key is having a team leader who is an ironclad star performer and is far more superior in technical and interpersonal skills than his subordinates in order to quickly gain their respect, and therefore for the ship to sail smoothly. I place even more emphasis on interpersonal skills and “emotional intelligence” than on technical skills.

Otherwise, you’ll meet endless time wasting debates and objections among the lines of why technology X is better than Y and why technical approach J is better than K.

Market Value Awareness of Software Developers

One of the consequences of not having a strong community is that software developers in Egypt are unaware of their true value in the market. Salaries are treated like military secrets, with many companies having oppressive measures to prevent this kind of information from leaking. As a result, many would work for far less than they can actually make. 

How much you earn actually boils down to how appealing and relevant your experience is, and your ability to negotiate. But that’s a universal assertion, it applies whether you’re in Egypt or anywhere else.

A common mistake made by inexperienced software developers is to answer the unethical question commonly asked by HR departments in the interviewing phase:

What was your monthly salary in your previous job?

That’s an unethical question, and it even makes me feel kind of offended. The best answer would be a polite “Sorry, but that’s none of your business.”

This question shouldn’t be answered. It corners you in either lying or telling the truth. The only respectable third alternative is not to answer it. If you have a problem telling someone that it is none of his or her business or feel its a bit aggressive (and it is!), you can answer instead:

Why don’t we discuss how much is a fair salary instead?

Closing Remark

I’ve written so much already, but there’s even more to come. Stay tuned, and provide me with your valuable feedback meanwhile!

Can you explain that to a 6-year-old?

One famous quote of the legendary Einstein goes “If you can’t explain a concept to a 6-year-old, you don’t fully understand it.”

The genius of this quote is that it forces you to unleash the power of analogy. And not just any analogy. Analogy with concepts and relationships that are clear, simple and vivid enough to be understood by a 6-year-old. It’s amazing how doing this might help us gain insight and vision into complex subjects. This quote simply implies that any complex subject can always be reduced to a simple analogy or example. But is it always possible? May be.

In all cases, only if you have enough understanding of a topic you then will be able to simplify it to the desired degree. This activity sparks your mind’s associative powers to find a simpler, similar set of concrete concepts from your everyday life. Not only can that lead you to better understand the topic, but even can let you uncover new insights into the original complex problem that you’re trying to explain. While you’re thinking about the analogy, your mind would use these thoughts to reflect on the original problem. This may lead you to find astonishingly simple solutions to complex problems, only because you have used a proper analogy to filter out the unneeded complexities of the original problem. I’ve experienced this many times myself.

All in all, I found this quote to be a great test against which I can test my own knowledge and insight about a subject – or even simply a technique to obtain a fresh perspective on a challenging problem. So… can you explain it to a 6-year-old kid?

5 Serious Learning Tactics for Tech Junkies

This article carries at its heart the purpose of helping you raise the level of your technical competence. It is about helping you make every second you spend learning something count; all using experience-grown tactics that prove their worth in practice.

Essential Introduction: Mental Models

Simply stated, a Mental Model (a known concept, not a fancy term I pulled out of my sleeve) is a certain imagination or logical explanation one has for how something happens or works.

For example, you were able to learn how to drive your car because someone helped you learn a mental model that explains the car’s behavior. This mental model was encoded in your mind in the form of a set of facts and cause-effect relationships.

“Shift the gear stick to D then push down the gas pedal and a car will move forward”,

“Push down the brakes pedal and a car will slow down to a stop depending on how far you push the pedal”

… and the rules go on.

These rules in collection plus some facts (what is a car, pedal, etc.) represent a simple Mental Model that if you hadn’t acquired through both learning and practice, you wouldn’t have been able to drive your car.

It is important to mention that some whiz professor called Dr. Donald Norman describes the properties of mental models – that they can be “contradictory, incomplete, superstitious, erroneous, unstable, and varying in time.”

Strategy… and Tactics

It can be argued that the purpose of education in general is learning Mental Models that are stable (not easily forgettable), reliable (can be recalled fluently), and useful (has value because you can perform something with). The following tactics are all geared towards helping you efficiently learn useful Mental Models. They help you get the best out of any learning experience you go through in everyday life.

With these tactics…

1. You’ll learn more in less time
2. You’ll remember more of what you learn
3. You’ll gain better insights from problems and challenges you face

Tactic #1: Do Most of Your Learning By Example

Hardly a surprise. When learning a particular technology, minimize reading and comprehending long theoretical text to the least possible. Be selective of what you learn, and start by the most common scenarios for applying the subject technology (taking JSF as an example, I’d start by creating an application with some simple forms, validation, templating, etc.). Practice those common scenarios to get hold of the basic concepts of the technology.

The way I see it, learning is a means to an end. Your ultimate purpose is to get some job done, learning the theory behind what you do is often enjoyable yet sometimes optional outcome.

Tactic #2: When Facing a Problem, Step Back and Reflect FIRST

Before jumping your browser’s toolbar to google your problem, do some careful thinking about it first. Although this might seem counter intuitive, the reality is that some proper focused thinking in your problem will certainly lead you to a better understanding of your problem.

The traditional approach is that you only have some superficial knowledge of the problem and may be some leads — may be in the form of a handful of error message or keywords. If you take those and you google them… presto. You often find a solution to your issue within a few clicks. Ironically, you have never really understood neither the problem nor its solution!

The bad thing about googling the hell out of every issue we face is that it often cancels out a very important intermediate step in any problem-solving effort: understanding and gaining insight into the problem. Careful thinking is an essential part of any serious problem-solving; do it when you want the problem and its solution to stick in your head.

Tactic #3: Learn More In Less Time By Condensing Your Understanding

Any complex creation of the human mind (e.g. technology or book) can be traced back to a few source concepts. These source concepts are the seeds around which the complex creation was formed. When reading long text or understanding a technology, “Step back and reflect” to condense your understanding into few key points, actively seeking to uncover those basic principles or concepts behind whatever that you are understanding.

Tactic #4: Learn Efficiently By Contrasting Close or Similar Concepts

Understand by contrast. If you spend one week learning a technology like Tapestry by example, and another week learning JSF by example, chances that you’ll end up understanding a whole lot about the essential concepts of View frameworks in general by just contrasting the differences between the two (more or less) similar technologies. Bonus: the contrasting happens in the background of your mind while you’re learning the second technology; you don’t spend any conscious effort doing it!

A simple yet extremely powerful variation of this tactic is (for example) to put yourself in the shoes of the creators of Technology X that you want to understand. What would they have done to implement Feature Y? If you do that kind of thinking before you attempt to understand Feature Y, you’ll be contrasting your own mental models of Feature Y against how Feature Y was actually implemented!

Learning by contrast is a very powerful technique. Use it whenever possible.

Tactic #5: Avoid Learning Without Purpose

In order for the Mental Models that you learn to persist and not evaporate, you have to associate them with some purpose that is important to you. Try reading a book about Hibernate just for the purpose of learning Hibernate… and then try to read the same exact book for the purpose of creating a small Library application, interleaving practice with reading. See what I mean?

The True Technical Architect

“Architecture“… what a grand word. Grand words with grand meanings emit a mysterious aura of sophistication and elegance. I won’t deny it. I, myself, fell in love with that word.

The Architect of “The Matrix” is pictured as a calm old fellow with the wisdom of Buddha and the profoundness of Shakespeare. That’s how Hollywood (read: people) perceives software architects… ahem, well, that’s not so far from the truth.

Many times, I see “Technical Architect” on business cards and in resumes. But what does that title or role really mean?

When there is ambiguity, I go back to the basics for an answer. That’s why, in a humble attempt to make things clear, I try in the following paragraphs to meticulously define the concept of “Architecture” and then move on to sketch a rough drawing of the true architect like I picture him or her.

Architecture… of What?

That’s the first wise question one should ask. I like to poke “Architects” and ask them… what do you architect? what kinds of decisions do you make? I either get back a blank stare, or another question plus some pseudo-random output: “what do you mean? I just do it. High-level stuff. I pick technologies.”

Well, we – developers and software engineers – build systems. But let me be more precise. We actually develop Applications. We might also integrate our Applications with other Applications into a System of Applications (think your web app, DBMS, FTP, Email, and BlaBla servers for example). These applications we build and integrate do not run in vacuum. They run, possibly, on top of other applications (think containers) that run over platforms (think Java) which run on operating systems which, in turn, run on hardware. Applications may also need to communicate over networks.

The union of all the above mentioned elements represents an IT Solution – a wild beast, and a complete answer to your customer’s business need or problem.

Great. Now, let’s define “Architecture” In the context of IT solutions. Architecture is essentially “Design” — only a high-level form of it. To design something is to…

…plan something for a specific role or purpose or effect; “This room is not designed for work”

The True Architect

So basically, to call myself an “Architect”, I should have had planned – at least once – either an entire IT solution or part of it for a specific intended role or purpose or effect. If that sounds easy, think again.

The very first purpose of any kind of IT solution is to be functionally correct and functionally complete; i.e. the solution (including all custom-developed applications) completely and correctly fulfills functional requirements, plain and simple.

But that’s just the beginning. All the “neat” tricks and conquered challenges behind great architectures (such as Google’s) were direct results of ruthless non-functional requirements and constraints – Performance, Scalability, Security, and Availability to name a few. Fulfilling those requirements is therefore an essential part of overall IT solution architecture. How to do that under different project constraints (available budget/time/ resources, imposed technologies/platforms/operating systems, etc.) is the name of the game.

That’s on the solution level. On the Application level, amongst the most important non-functional requirements are Maintainability, Extensibility, and Configurability. Architectural patterns such as Model-View-Controller (MVC) for example became so popular because they facilitate maintenance, extensibility, and configurability of otherwise spaghetti-coded web apps.

Another important aspect is that technologies bring along their limitations and performance characteristics. That’s why choosing technologies that most fit the purpose of the application/solution is one of the most fundamental and far-reaching decisions in IT solutions development. Technology is always a means to an end.

Bottom line

So anybody who did application or solution architecture once can mark himself as an architect. However, a true architect who earns my respect is one who has lots of real-world experience in meeting non-functional requirements in ruthless production environments. A true architect is someone who has developed a rich tool box of principles, patterns, and anti-patterns of solution and application architecture.

Boost Your Productivity Using Apache Tapestry

If you are like me, you really don’t have time to waste with frameworks that suffer from favoring “elegance of design over everything else”.. take the old EJB model as an infamous example.

Personally, all what I care about is finding new ways to improve productivity. Nowadays, I prefer frameworks and APIs that are designed for simplicity of use in the first place. You invest some time and effort, and gain back generous increases in productivity.

If you’re into Java-based web applications, you probably know how challenging it is to develop the UI Layer of the application — while keeping View code clean (readable) and neatly separated from back-end code.

That’s why a number of months ago I decided to go on the hunt for the ultimate UI framework — if it ever existed. I pictured a framework where complexity of handling UI events and moving data back-and-forth between Model and View is hidden from the developer, such as with desktop applications.

I enumerated a number of frameworks from java-source.net. I did not consider some excellent technologies such as AdobeFlex and another framework called ZK (watch out for that one) just because you do not write the interface in HTML or any close markup, but rather in completely different proprietary language.

I also ignored frameworks that do not have an online demo. Presence of an online demo is the least thing framework owners should offer to their prospective users… lack of an online demo gives me negative vibes about the health of a UI technology.

I ended-up looking at a couple of JSF implementations, Apache Tapestry, and Apache Wicket based across several dimensions:

• Learning Curve
• Maturity (which reflects on other traits such as reliability, security, and performance)
• Programming Model
• AJAX Support

The following are the results of my quick research. This is not an extensive research, with anything but concrete results. I was in a rush (as always!). I just brushed over the surface, with high reliance on first impressions and short test drives. Use the results with caution!

I easily ruled out JSF. I got the impression that it was plagued with the same “elegance of design over everything else” virus that used to plague Java specs (the situation is getting better now). As usual, JSF needs tedious XML configuration. Plus, web pages are written in JSF markup which is a turn-off to me. Yes, I don’t care that JSF is a specification.

There was a number of open-source implementations: Apache MyFaces and IceFaces. Only IceFaces seemed to be mature and impressive enough anyway. The documentation of IceFaces was far from complete or accurate, and it seemed to me that in addition to the learning curve of JSF, there will be another trickier learning curve mounted over it… to learn the “secrets” and internals of IceFaces (that was with version 1.5 as I recall, now I’m seeing a version 1.7 so perhaps things have changed).

What really clicked with me were Apache Tapestry and Apache Wicket. There is some similarity between them; they are both “component-based” frameworks that map parts of the UI in the HTML pages 1-to-1 to Classes and Objects on the server. Component-based frameworks are far more easier to deal with than JSPs and other frameworks that expose the cycle of request/response handling (such as the old Struts). Tapestry hides the request/response handling cycle pretty neatly.

So, I gave Tapestry a try. Although the documentation is not the best that they could offer, it is very extensive. The quick-start guides delivered a really quick start. I actually was able to write a simple application in a couple of hours, and boy, I was impressed with Tapestry’s potential. We’re talking super-productivity here once you get over the initial (relatively shallow) learning curve. Add to that the following great advantages:

• You write your Web Pages in pure HTML (now XHTML with Tapestry 5)… Tapestry’s markup is elegantly hidden within HTML markup.
• Moving data from your business logic to the web UI and back is as easy as populating and extracting data from POJOs using setters and getters.
• Built-in AJAX support, DOJO and Scriptaculous are integrated.
• It is easy to build your own components and extend the framework.
• Page navigation is easy and intuitive… no need for lengthy XML files as in JSF.
• Persisting data into a Session is a matter of calling a “set” method on an annotated POJO field.
• Tapestry adopts the convention-over-configuration philosophy. No piles of XML configuration files, just put the jars in your Lib directory, add a couple of entries in your Web.xml, and everything just works.

After I experienced Tapestry, I was quite impressed and decided to skip Wicket and get started with Tapestry immediately. Tapestry and Wicket are more or less similar, only Tapestry is more mature. However, Tapestry has its flaws as well, ones that I recognized when I got up close and personal with the framework. So, if you decide to use Tapestry, be ready to live with the following issues:

• First, you have to decide which Tapestry version to use. 4.x and 5.x are completely different and there is no backward compatibility. 5.x has been in “beta” stage for a long time now, but as far as my experience goes, it is stable. 5.x is far more simple than version 4.x due to its streamlined design and the use of Annotations.
• The documentation on the website is extensive and sufficient… but as soon as you start meeting more complex scenarios, things start to get a little darker… and in some cases, you might need to do the occasional dive into the internals of the framework to understand how things work.
• Tapestry 4.x is not satisfactory on the performance side due to the design of its request/response handling cycle. 5.x is much better on the performance side.
• For some reason, the releases of Tapestry are few and far in between. I don’t know why. That framework has lots of potential.
• The only book on Tapestry covers 5.x and seems to be significantly outdated. Many of the book’s examples are obsolete and do not work with the latest release of 5.x. So that brings us back to rely on online documentation.

Bottom line: Tapestry is a great framework that boosts productivity, a framework that has been elegantly designed and written for the sole purpose of making your life easier. The cons of Tapestry can be lived with, and the framework’s clean design makes it easier to understand its internals. You’ll enjoy working with Tapestry. Kudos to the Tapestry Project team.

5 Keys to Great Requirements Documents

Critical activities in any software project heavily rely on the presence of high-quality up-to-date business requirements.

Architects base the architecture and design of a system on requirements. QA draw their test plans from requirements. Developers start working off documented requirements… that how it should be at least.

So bottom line, you will have to record requirements in one form or another. This applies even if you’re an agile process supporter who believes in light documentation. The following tips can and should be adapted to suite your documentation practices.

Prototype First

Frankly, I don’t know how anyone could NOT do this step. Prototyping really simplifies requirements documentation. The prototype usually becomes the most important part of requirements, for the following reasons:

• It minimizes the inevitable risks of requirements change, and gives your requirements much wanted stability. Users start to be enlightened and understand what they really want as soon as they have some working prototype to interact and play with.
• Screenshots (an output of basically any prototype) are essential parts of Use Cases or User Stories; they reduce the amount of writing required to the minimum… you just include the screenshots and reference them.

Prototypes can be as simple as drawings and sketches on white paper, more complex and interactive like “paper prototypes” or can be done using a specialized prototyping tool.

But the prototype is only the beginning of the story. While prototyping, you should start planning how you will document the requirements represented by the prototype.

The next step is to…

Know and Honor Your Audience

In all documentation I’ve seen and examined, this is one of the most ignored fundamental principles. You have to understand who you are writing for, and then write to give them what they expect.

First and foremost, you write for your customer. So pay them due respect and:

• Understand your customer’s lingo and use it.
• Write in the customer’s preferred language, if possible.
• Organize requirements in the document by priority – the most important to customer first.
• Use simple diagrams and illustrations extensively – customers really dig pictures way over long boring text.

Don’t forget the rest of your audience in the project team. Understand what they expect and take their input into consideration.

Never Start with a Blank Page

Templates exist out there for a reason. One of the greatest opportunities in documentation writing is to re-use the knowledge and experience gleaned by other people in the form of templates.

Starting with a template can give you a great jump start. But almost always, any template needs customization to fit your needs. Strip a couple of sections, add a few, and change the name of some others until you are satisfied.

The point of customization is to create a simple and concise document. You have to question the presence of each and every section in your document. Ask yourself questions such as: is that section really relevant? Who of my audience will find it interesting and why?

Great templates that I’ve used myself include those of the Rational Unified Process (RUP).. you can also take a look at the templates of the open-source ReadySET project.

Pick a Style and Be Consistent

“Consistent” here must be in everything – from your writing style of Use Cases and your use of terminology to the use of diagrams. Consistency makes it easy for anyone to quickly scan the documentation for a piece of information… not to mention being a fundamental trait of any professional document.

Be Prepared For Change

You must have heard it thousands of times: the most certain thing about requirements is that they do change. If you do not keep up with the changes, the requirements document loses its value and fails its purpose.

That’s why you have to be careful while picking your prototyping technique. Is it easy to update your prototype?

But what should you do when requirements change? Do you go back and change the requirements document? Yes and no.

Yes, if the requirement item (use case or feature description) has not yet been implemented. You go update your prototype and your requirements document.

The answer, however, is No when you have already implemented these requirements. At that point, the system itself manifests the requirements. So you go update the system. But that will leave the requirements document outdated, won’t it?

In my opinion, this is fine – as long as these changes are tracked in an issue tracking system. That way, you can spare your effort and “batch update” the requirements document if you ever needed or wanted to.

Review and Refine with Your Customer

After you’ve produced something, don’t just email it to your customer and expect them to review it or even read it carefully. That ain’t happening. Except in the most motivated of customers, the correct way to do it is to go in person and review and refine it with them.

Oh, and on the way, arm yourself with the patience of an ant — you’ll need it.