After AWS Re:Invent, I spent some time setting up AWS Cloud9 service. I was a user of Cloud9 before Amazon acquired them. I really like the IDE and I was wondering how it integrates with the rest of the AWS services.
Then I did some more learning and setup and here are the results:
You can use AWS CodeStar service as an orchestrator/workflow that allows you to:
(1) Code an application (different templates with different languages) using AWS Cloud9.
(2) Manage the source code via AWS CodeCommit.
(3) Deploy it using AWS CodeDeploy.
All of this is managed through the AWS CodeStar dashboard. As part of creating a project within AWS CodeStar, I had an option to set it up with just one EC2 or with Elastic Beanstalk. For simplicity I chose the EC2 flavor and successfully deployed the “Hello World” Python Flask application using AWS CodeStar. After I deployed the initial version of this Python application, I went back into the Cloud9 IDE and changed some code. After doing the “git push” to the master branch, I went back to the AWS CodeStar dashboard and I was able to see in the flow that CodeCommit and CodeDeploy steps automatically started and the code changes were deployed within short time. The URL of the Python web API was provided in the AWS CodeStar dashboard and after browsing to it, or after trying it in Postman, I was able to see my changes.
Try it out because this is how the cloud computing should work; we need to focus on the business logic of our applications and the cloud providers need to take care of the rest. Obviously the AWS CodeStar is the beginning because it is focusing on setting up web applications but the setup of other peripheral services is still not included. For example, if I wanted to set up some SNS topics and SQS queues, I am not sure if the CodeStar would manage that for me.
I am looking forward to future AWS CodeStar enhancements.
P-BAR P = Product value B = Bug fixing value A = Architecture value R = Robustness value
There are many different ways to measure the productivity of your agile teams, but certain measures get typically forgotten but they are ultimately why you are doing the work in the first place. That's why I thought of this measuring system called P-BAR.
P for Product value and it is purposely the first letter. This is what ultimately matters. This is what your end customers benefit from. In a perfect sprint, you are always adding a lot of product value. However, the real world is not perfect and that's ok. It is ok not to have a lot of product value in sprint 0 and 1. But later you should be having sprints with the P value high.
Bug fixing value makes sense after you some number of P-value sprints and now functionality is being tested, and it is ok to have some percentage of bugs being fixed. However, if you are adding a lot of B-value in the 11th hour of the project, then it means you left all your bug discovery and/or fixing for late in the game.
Architecture value within your sprints may be high at the at the beginning sprints and occasional sprint later, but you should question it if too much of this being added closer to the end of your releases.
Robustness value measures to what degree you are developing your code to the enterprise level. This is something that should be evenly distributed throughout all sprints. It should be part of the culture and not something being done at the end.
In this article I will not get into how you measure this. You will somehow measure each value for each sprint and you can use it to detect if you are heading in the right direction and you can also capture trends/averages for overall release. At some point, I will write a full article about this.
Microservice Architecture, but NOT microservice mindset?
We are on the same team!
Let's say we as a company decide to implement the microservice architecture and we also decide to split the teams into domains for each team to own a domain (group of microservices).
You may be a member of team A and I may be a member of team B and we are all on the same ship. There is a water leak in your area of the
ship, but my side of the ship looks good.
What do we do?
Let's help out each other because we are all members/passengers on the same ship.
We are all going to the same destination :)
Microservice teams were created to pave domain boundaries at the architectural level, but it does not mean that we should create boundaries among members of different domain team members. Yes, the API contracts are definitions for how our services communicate, but for the sake of final destination, we need to tear down those boundaries when it comes to approaching each other for help and proactive integration.
It is beautiful to see ONE team in action especially in crunch time. Almir Mustafic
Python-like or C-like as the language to introduce programming to students in high school and university?
The question is: Do you introduce programming concepts to high-school/university students using languages that handle memory and other things for you or do you start introducing all of these concepts in languages like C that require you to understand all aspects.
I will tell you what worked for me. I was introduced to programming in grade 10 using Basic programming language. There was a version called Better Basic and also Quick Basic. Then in grade 11 we learned procedural programming in a programming language called Turing (not Turing machine but a Pascal-like language developed by University of Toronto for teaching purposes). Then a year later, I started getting interested in C and C++. As you can see, I eased into the languages that introduced me to NULL exceptions and memory leaks :)
With this approach I was not overwhelmed and this set up the foundation for a fun journey in the software engineering field. However, everybody is different. What would work for majority? Almir Mustafic
Software Engineering — Building layers of the cake instead of slices of the cake. In software development you could go into separate rooms and build slices of a cake and combine it all into a single cake hoping to have a good cake.
The following alternative is better. Build layers of the cake and that will naturally lead itself to early integrations, early visibility to software functionality from the product management and the end user point of view.
Should you design your system with defensive Plan-B mindset or should you keep it simple thinking optimistically? I don’t think the answer is yes or no.
I am talking about the case where you prefer plan A, but the confidence level is not there yet. I think you can design it clean and simple with with hooks in your code so you can harness or feature toggle the plan B functionality. If plan A is chosen then all you need to do is clean up the hooks for the harness or feature toggle that was not needed. Nothing to lose, but you need to have the discipline to clean the unnecessary code when the dust settles down.
Imagine the world in which feature branches don’t exist. Imagine the world where you don’t delay your code integration (code merges) until 11th hour and you deal with it on daily basis. Imagine the world of just main trunk development in GitHub and development mindset and development capabilities that allow your teammates to see your code sooner rather than later. Pay me now or pay me later is what one of my friends has been saying.
Yes, this goes against what everybody is doing out there, but it is worth exploring. It requires the right mindset and technical skills. It ultimately boils down to the following advice that I always give to fellow developers:
* Regardless of what branch you are in, when you check in or commit your code changes, pretend that these changes will go into production right away. Then ask yourself a question if my code will negatively impact customers and will it change the customer experience if product team wasn’t intending to do so yet. If the answer to above question is Yes, then you are not doing it the right way. Did you stub out your code or used feature toggle approach?
Programming is an act of giving a machine some instructions so it can perform things repeatedly. Software engineering is an act of programming and everything else that goes along with it in order to deliver enterprise software to production.
Within software engineering, the more time you spend coding your applications and less worrying about DevOps things and processes, the better for you. Make sure you are heading in the right direction.
Let’s say you use a cloud platform to run your applications. The whole point of a cloud platform was to abstract things for developers in order to spend more time coding applications and less worrying about how to set up the resources in the cloud platform. So if you are hiring a lot of DevOps experts, then it means that the platform itself did not abstract the cloud enough.
In conclusion if the act of software engineering is close to the act of programming, then you basically reached that ultimate point where your technical teams are spending more and more time coding and less setting up and managing the resources in the cloud platform that the cloud platform is supposed to handle for you.
Imagine the world of software architecture where every POC becomes a production software without changes. That’s one extreme.
Now imagine the opposite where no POCs get done and no free innovation happens and everything has to be 110% planned and build perfect from day one.
If you and your team can take the good aspects of both extremes and combine into one nice approach, then you will innovate and that innovation will turn into enterprise worthy software.
As a software engineer out of college, your initial goals are to get use to the professional working environment.
Once you get passed that first hurdle, then it is all about the programming or coding skills. You want to improve your .NET C# skills or Java skills or NodeJS skills or Python skills or database skills. You want to learn the frameworks around these programming languages better and you want to learn different ways to optimize your code and apply different design patterns in your implementations. These are great skills to have and in a lot of cases required skills to have. The most important thing is that you are enjoying what you are doing.
At this point you have reached that level that will give you the right to label yourself as a great coder or a great programmer. However, there is still more to go. At this time your main focus is programming and you may be lucky that your technical lead and software engineering manager are abstracting a lot of things for you and shielding you so you can focus on programming most of your work day.
Don’t feel rushed that you need to advance vertically in your career. You need to feel comfortable where you are, and you need to follow your gut feel. When it’s time for you expand your horizon, it is your choice to make. It is all part of becoming a great software engineer.
Now it’s time. Willingness to listen and learn from other senior software engineers, tech leads, solutions architects and software engineering managers is the key to turning yourself from a good coder into a good software engineer. This is the type of stuff you cannot easily google. You need to be patient. You have to approach it with open mind and realize that there is no book “Experience in a Nutshell” that can help you expedite this process. After some time doing things the right way becomes the second nature. In fact, it later becomes harder to NOT do things the right way.
What you will learn is:
* What it takes to develop and deploy robust applications at the enterprise level to production and then maintain it in production.
* SDLC (Software Development Life Cycle)
As you learn more and more and software engineering concepts come as second nature to you, that’s when you know you graduated and joined the club of great software engineers.
Thank you for reading. Please follow me here on Medium.com or check out my personal site: http://www.almirsCorner.com
The perception is that software engineering only care about the low-level details and that’s where they find their motivation.
Let’s go to Day 1 out of college and you have a first professional software engineering job. It is overwhelming just to deal with the reality of having a professional job and the junior software engineers are generally all about solving the low-level technical problems and being motivated by the technology that is used in problem solving.
Later in your career as a software engineer, you learn the business better and then you start understanding what it takes to deliver an enterprise-level application to production and keep the end customers satisfied. That’s the turning point in your career where you find the motivation in:
* Optimizing the code and working on other geeky things
* Understanding how your application or your module or your few lines of code relate to the overall revenue numbers and customer satisfaction.
Then even further in your career you may get to the point where programming languages, libraries, APIs are just tools or means of getting a stable and robust application for our end customers. So your motivation may mostly come from “problem solving” in general and getting cool products to our customers.
After saying all of the above, we conclude that software engineers do care about the business value.
We as software engineers are blessed to have opportunities to work in this field.
I am so happy to be able to share my experiences with you through my blog posts, but I also decided to post video about different software engineering topics.
Here is the introduction video of my YouTube series:
A lot of times, as the technical team in a company, you get pressured to build a self-service tool so that business/product-team can perform their configuration and content changes without involving the tech team.
Should you just build what the business/product teams requested or should you justify why this should not be done in the interest of both the business and the tech team and ultimately in the interest of your end customers?
To question the requirements from your product/business teams, you need to have very good justifications and counter proposals. In a good work environment, the requirements should not just come from business and be executed by the tech team. It should be a two way conversation. The perception is that the tech team just wants to deal with technical details, but the reality is that the tech team wants to be involved at the idea phase before the requirements get baked and turned over for design and implementation. Early in my career I was happy with just focusing on technical details and enjoying the process of solving problems through coding. Then later I realized that my motivation comes from knowing what the business team wants to achieve and for me it meant that I was able to translate that to a much better working software.
Let’s use the Price & Product Configuration and its UI tool as an example. This subject is very dear to me and I have certain vision that I would like to share in order to get to the essence of this article.
Let’s assume you built a set of microservices and database structures behind each service to support the following capabilities:
* Creating offers (no UI but configurable) and specifying the price
* Creating benefits/features (no UI but configurable)
* APIs that allows you fetch offers and query offers in order to decided what offer(s) to display to customers
* API that allows you fulfill offers and ultimately create customers’ subscription records
Let’s say these feature are very stable in production, but you still need to continue to iterate and build new capabilities and enhance existing capabilities for the business requirements that are geared towards the end customers.
At the same time you maybe getting business requirements that are are about the self-servicing of these features for internal customers (your product and business team). These requirements are all about above mentioned capabilities of microservices and introducing some UI tool that allows the business team to perform price and product configuration (creating offers, creating benefits, configuring and changing prices for offers, …etc).
The goal purely looking from the tech team’s angle is to have agility and to innovate. When you introduce UI tools on top of your evolving microservices, then it really slows down your ability to innovate and adapt quickly. Why does the building of a UI tool slow you down? Think about the possible impacts if you change the underlining functionality and structure of your microservices:
* You will also need to change the UI to accommodate for these changes
* You will need to possibly train the users of UI if the changes are drastic enough
* And you will need to roll this out without causing too much disruption.
All of this requires a lot of effort and coordination. This effort and coordination is acceptable after you have a mature product in production, but it is too much overhead if you are still rapidly evolving.
If you have a choice, you really need to consider delaying any self-service UI tools for your business until the data structure and microservices for your data structures are mature enough; otherwise, you would be just slowing down the delivery of new features in these microservices for end customers even though the tech team can deliver all the changes that the UI tool would do for business users by just making application config changes. This is a very good alternative during this crucial innovation phase.
Therefore, as the tech team you need to be able to convey this message to your business and propose investments into the capabilities of these microservices for end customers before building a self-service UI tool. But at the same time, you need to provide to business some visibility to the configuration from these microservces so they can better visualize what what capabilities they have so far. That could be just a BETA version of some simplistic UI tool that is about purely REAL-ONLY information . If it is a “read-only” version of this tool, then the effort is much lower and there is less to consider from the security point of view. Overall, I think this is a good compromise for some amount of time to keep the agility at a high level and to keep the innovation going.
Let’s go back to the days when Google released the beta version of GMail. They kept the beta version for quite some time and they informed users that they don’t have any obligations to support customers requests. Why did they do this? They did it because they did not want to slow down the innovation of GMail in its early days by introducing the formal UI; that’s why kept it as a beta version. I am not saying that you should be doing this for as long as Google has done it, but the overall conclusion is the same.
In conclusion, consider delaying building the official self-service UI tools for your product/business users in order to keep the innovation and agility at a high pace during crucial phase in the evolution of your microservices and data structures/configurations.
Thank you for reading this article. Please follow me here on Medium.com or check out my personal blog: http://www.almirsCorner.com
Code is just code. Process is just process. However, when you combine the two, they impact each other in ways that they should not.
Let’s assume you built a platform for your team. Then the process came later and slowed down things but kept things as stable as possible. Then software engineers realized that they can follow the process and do the development in such a way that they can go quicker through this process. I am talking about writing code that is so over-configured that it actually allows you to deploy changes to production quicker just because it is a configurationchange that you are deploying. When your platform is over-configured, the weight of a configuration change is so huge that you can introduce functional changes to the degree that makes you wonder if you are adding so much risk to the business because you are taking a shorter path in the process workflow. The risk is there because software engineers don’t feel comfortable making these heavy-weight configuration changes and nobody likes doing any change in the engine of the code that drive this heavy configuration.
This really smells as coding to get around the process. Is this right?
I am all about automating the process, but when it comes to over-configuring your code just for the purposes of getting around or going quickly through the process, it is totally wrong.
As a software engineer or a technical lead, every time you find yourself thinking how to adjust the code to make it very configurable to get around the process, you should ask yourself the following questions:
How can I help automate or implement the continuous integration in such a way that I can do the right thing in my code. The right thing could be that you actually make a code change that requires a compilation, build and deploy and the code instead of trying to make it configurable and complicating your codebase and increasing maintenance costs.
How can I work with the Change Control team and improve the SDLC/PDLC process in order to make it jive with the platform/framework you have implemented in your company.
It is ok to have a simple code change instead overcomplicating the code to make it configurable so that you change ends up being a configuration setting.
In conclusion, it is important that you design your process at the same time you are building your new platform. That’s how they compliment each other better. As for software engineers, please don’t code to get around the process. Please code it right and identify incremental changes how you can improve your software deployment approach.
Thank you for reading. Please follow me here on Medium.com or check out my personal blog: http://www.almirsCorner.com
A lot of times software developers get caught in the situation where they are reaching for the book of design patterns to see how they could solve a specific problem.
Programming languages are rich enough to help you solve different problems and a lot of programming languages and its frameworks can be used to easily implement different design patterns. However, when something like this is readily available to you, you need to have enough discipline NOT to reach for it right away.
What do I mean by this?
It is important to first truly understand what problem you are trying to solve. Work with your product owner and make sure that requirements are true business requirements without inclining any type of implementation. Once the requirements are in good shape, then you know what problem you are trying to solve.
Then you need to work out different solutions for that problem. This is where you would use your overall software engineering experience. You should NOT open a book of design patterns and look for what could solve your problem, but you would rather work out a solution and then if it happens to be one of the known patterns, the implementation of it would be that much easier.
Please think about this any time you are presented with a problem to solve.
As plotting graphs is something that my oldest kid started doing in school, I decided to finally spend some time doing it in Python. This article is about the basics, but I will also later get into the calculations of medium, mean, mode and other more complex concepts of plotting graphs using mathematical formulas.
Here is an example that shows you the following:
* takes a list or more lists and plots it on the graph
* plotting of points on the graph with or without connecting it into lines
* applying the labels on the x-axis and y-axis
* applying the overall title on the graph
* adjusting your scale based on what type of data you have for each axis
Install the following libraries before running the code below. By the way, the example is written for Python 2.7.x:
Here is the example that you can play with as it has a command-line menu that allows you to choose which type of graph you want:
__author__ = 'Almir Mustafic'
from pylab import plot, show, title, xlabel, ylabel
from pylab import legend
from pylab import axis
def main():
print("main program")
while True:
print_menu()
choice = raw_input('Pick from the menu? ')
if choice == '1':
plotting_example01()
elif choice == '2':
plotting_example02()
elif choice == '3':
plotting_example03()
elif choice == '4':
plotting_example04()
elif choice == '5':
plotting_example05()
elif choice == '6':
plotting_example06()
elif choice == '7':
plotting_example07()
elif choice == 'q':
break
print("")
def print_menu():
print("=========================================================================")
print('1. Plot a few points WITHOUT dots for the given point and connect as line')
print('2. Plot a few points WITH dots for the given point and connect as line')
print('3. Plot a few points WITH dots and WITHOUT the LINE')
print('4. Plot Ladera Ranch town temperature with DOTS and LINE and NO X-Axis specific range')
print('5. Plot Ladera Ranch town temperature with DOTS and LINE and X-AXIS range specified')
print('6. Plot Ladera Ranch averages in 1996, 2006 and 2016 for 12 months. 3 graphs')
print('7. Plot Ladera Ranch averages - Adjust AXIS based on min and max')
print("q. QUIT")
print("=========================================================================")
def plotting_example01():
print("Plotting example 1............")
x_numbers = [1, 2, 3]
y_numbers = [2, 4, 6]
# Plot the line WITHOUT dots for the given points
plot(x_numbers, y_numbers)
show()
def plotting_example02():
print("Plotting example 2............")
x_numbers = [1, 2, 3]
y_numbers = [2, 4, 6]
# Plot the line WITH dots for the given points
# Markers can be 'o' or '*' or 'x' or '+'
plot(x_numbers, y_numbers, marker='o')
show()
def plotting_example03():
print("Plotting example 3............")
x_numbers = [1, 2, 3]
y_numbers = [2, 4, 6]
# Plot the line WITH dots and WITHOUT LINE
plot(x_numbers, y_numbers, 'o')
show()
def plotting_example04():
print("Plotting example 4............")
ladera_temp = [53.9, 56.3, 56.4, 53.4, 54.5, 55.8, 56.8, 55.0, 55.3, 54.0, 56.7, 56.4, 57.3]
# Plot the line WITH dots and the LINE
plot(ladera_temp, marker='o')
show()
def plotting_example05():
print("Plotting example 5............")
ladera_temp = [53.9, 56.3, 56.4, 53.4, 54.5, 55.8, 56.8, 55.0, 55.3, 54.0, 56.7, 56.4, 57.3]
# years = range(2000, 2013)
years = range(2000, len(ladera_temp) + 2000)
# Plot the line WITH dots and the LINE
plot(years, ladera_temp, marker='o')
show()
def plotting_example06():
print("Plotting example 6............")
ladera_temp_1996 = [53.9, 56.3, 56.4, 53.4, 54.5, 55.8, 56.8, 55.0, 55.3, 54.0, 56.7, 56.4]
ladera_temp_2006 = [43.9, 66.3, 46.4, 63.4, 34.5, 75.8, 46.8, 65.0, 75.3, 64.0, 56.7, 46.4]
ladera_temp_2016 = [23.9, 26.3, 36.4, 33.4, 44.5, 55.8, 66.8, 75.0, 65.3, 54.0, 46.7, 36.4]
months = range(1, 13)
# Plot the line WITH dots and the LINE
plot(months, ladera_temp_1996, months, ladera_temp_2006, months, ladera_temp_2016, marker='+')
# OR you can plot separately like this and the final show() method will know that it needs to plot everything that is queued up.
# plot(months, ladera_temp_1996, marker='o')
# plot(months, ladera_temp_2006, marker='o')
# plot(months, ladera_temp_2016, marker='o')
# Apply the legend to tell the graphs apart
legend([1996, 2006, 2016])
# Apply the TITLE, X-axis and Y-axis label
title('Average monthly temperature in Ladera Ranch town')
xlabel('Month')
ylabel('Temperature in F')
show()
def plotting_example07():
print("Plotting example 7............")
ladera_temp_1996 = [53.9, 56.3, 56.4, 53.4, 54.5, 55.8, 56.8, 55.0, 55.3, 54.0, 56.7, 56.4]
ladera_temp_2006 = [43.9, 66.3, 46.4, 63.4, 34.5, 75.8, 46.8, 65.0, 75.3, 64.0, 56.7, 46.4]
ladera_temp_2016 = [23.9, 26.3, 36.4, 33.4, 44.5, 55.8, 66.8, 75.0, 65.3, 54.0, 46.7, 36.4]
months = range(1, 13)
# Plot the line WITH dots and the LINE
plot(months, ladera_temp_1996, months, ladera_temp_2006, months, ladera_temp_2016, marker='+')
# OR you can plot separately like this and the final show() method will know that it needs to plot everything that is queued up.
# plot(months, ladera_temp_1996, marker='o')
# plot(months, ladera_temp_2006, marker='o')
# plot(months, ladera_temp_2016, marker='o')
# Figure out AXIS X and Y min and max and set it for the graphs.
print(axis())
xy_tuple = axis() # returns tuple (x-min, x-max, y-min, y-max)
my_ymin = xy_tuple[2]
axis(ymin=my_ymin-20)
axis(xmax=xy_tuple[1]+6)
# Another way to set X-min, X-max, y-min, y-max
# axis([xy_tuple[0], xy_tuple[1], xy_tuple[2], xy_tuple[3]])
# Apply the legend to tell the graphs apart
legend([1996, 2006, 2016])
# Apply the TITLE, X-axis and Y-axis label
title('Average monthly temperature in Ladera Ranch town')
xlabel('Month')
ylabel('Temperature in F')
show()
################################################
if __name__ == "__main__": main()
Here is the command-line menu that you should see when you run this Python code:
For example, if you select option 6 (graph that uses a lot features summarized above), then you will the following graph pop up:
If you select option 7, then you will see how the scaling on the graph is adjusted explicitly through code:
