you live and learn
Took me a while to write about this, but I really love Project Euler. The page is a collection of math challenges, which require some programming (I saw just one which could have been computed by a closed formula without any help). The first ten are quite easy to solve and are more commonly known math problems. Prime numbers and combinatorics play a strong role. But then the difficulty rises quite quickly. Usually it takes me one to two hours to write a Python solution for one. If I would – like I should – write unit-tests for each single method and not for a few selected one, then I guess 50% more.
It’s great: each problem is a closed, separate problem, which requires some algorithmic thinking and – of course – some proper implementation. If you chose the wrong path time or space complexity will kill your ambitions quite quickly. But proper solutions are computed most of the time in less than a minute.
Most of the time I rely on basic python structures and common libraries. But I’ve also given NumPy, itertools, etc. a try. Speeds up the process quite a bit.
My next goal is to fix problem 47, because then I’ve handed in solutions for all of the first fifty problems.
The highest challenge (with also the highest difficulty level (for me) so far) was problem 668. Due to a really big (80 GiByte!) boolean array the computer had a hard time swapping memory. So it took almost 36 hours to finish. By the way: less than 900 people worldwide have solved this issue #tinyflakeofpride
Of course, several geniuses have dedicated pages to optimal solution strategies. Which is a nice idea. But I avoid them. Most of the times stepping back, thinking without a display about the problem and if the chosen approach was a good one, is more helpful. A solution by cheating is nothing which renders any reward.
I’ve just spent the last eight hours attending a workshop about #SageMaker, #AutoPilot, #BERT, #Athena, #TensorFlow, #Spark, [..] and I am feeling a bit light-headed.
Of course, the talk and guidance given by @AntjeBarth and @ChrisFregly was really well prepared, but if you’re just a ML-beginner (like me) and if then over 9000 of new technologies drop, you have to work hard to follow the fast paced event.
Of course, I started my ML-journey in the summer of 2019, but it was more focussed on image-processing, not #NLP. I worked before with #Python, #Jupyter notebooks, TensorFlow and #Keras, but that whole SageMaker-thing was new to me.
And I see the potential: instead of running the stuff locally, you prepare, prototype and run your ML-app inside Amazon’s infrastructure. And that AutoPilot, which helps to quickstart the prototyping by trying several preprocessing-steps and models for you on your data, looks promising. Will definitely give it a second look.
Notes can be found at: https://github.com/marcelpetrick/KubeFlow_BERT_GPU_TensorFlow_Keras_SageMaker_Workshop (need lots of polishing, as always)
Crazy times we live in! And I am thankful for this block of time on a weekend 🙏
Passed the exam successfully :party: Proof can be found here: https://www.certible.com/de/verify/fc832c0524c8711b8871e1c645944583
I’d like to thank Stephan Christmann for the preparation and my dear wife for the support.
Strive for quality incrementally and iteratively 🙂
#isaqb #cpsa #softwarearchitecture #youliveandlearn
Certible page with the proof.
Some of the profits from this seminar ‘practive acting before an issue would turn into a problem’, ‘strong cohesion, loose coupling’, ‘the biggest leverage for guaranteed quality after software engineering and testing is done properly: requirements engineering and software architecture’.
20200827 – day 4
Long recapitulation of the lecture from yesterday, then the last chapter about support tools and static code analysis. Afterwards some discussion about a real world example from one of the participants.
The early closure left enough time to prepare for the exam (scheduled friday 12:00).
The exam took place online, I had to prepare a totally clean desk, proof by moving the camera that there is no one else in the room and then solved 41 questions within a maximum time limit of 75 minutes.
20200826 – day 3
Sadly, I haven’t written about that day directly afterwards, so the summary will a bit shorter: recap of day 2, then we learnt about metrics and FMEA and risk-evaluation. Then we solved and sicsussed 37 test-questions for the exam. Of course, they were not identical to the official questions. But came quite close 😉
20200825 – day 2
Day started with the homework-questions we had to prepare and some recapitulation. Works well, but I am already a bit overwhelmed of how much is there to learn. The more you see, the more you know, you know nothing. For my whole life I’ve done monoliths on different OS and with different technologies, ok, twice some message-based architecture. But there is so much more. I should definitely get myself more invested in server-architecture, REST and microservices. Today we covered architecture patterns and approaches, how to reduce dependencies, .. and did several exercises. BTW: I’ve forgotten to mention that I take digital notes as Markdown (see github). Again homework and much to learn and read. And my book “Zertifizierung für Software-Architekten” from Starke/Hruschka arrived.
20200824 – day 1
The seminar takes solely place in Zoom, trainer is Stephan Christmann and he does his job quite well. We are just a bunch of four apprentices, which is ok, because then the exercises (and there are some) are not crowded and we can utter questions whenever we like. Of course, the first day is loaded with definitions after some “soft” start into the topic. Looks like a software architect (abbreviated SA later on) needs more or less the same personality traits like a requirements engineer (see IREB) or a test engineer (see ISTQB): resilience, curiosity and communicative skils and a proactive mindset. We even got homework, yay!
20200823 – before the seminar
in the past, ive not done this. that i recapitulated much about everx seminar or workshop. if i did, then it was most of the time areview of the successful participation. but, reality is different. before something starts, there is the tesnion and the fear of not being well prepared, while doing there is the fun to lear some new insights, see your capabilities grow, learning stress, befor he test the testion is almost a t maximum and then the release. after seein a successfuly test rulst. htherefore i want to cover the following four days of this chapter of my learning journy.
Initial plan was to visit a course at the VHS (MVHS: Münchner Volkshochschule) about ‘NLP with Python & DeepLearning’ (natural language processing). But the tutor quit, so I checked what else I could learn! Notes and examples are archived here: graphicsWithPython
The course took place on two evenings. Lecturing person was Dr. Günter Spanner. We coasted through examples with matplotlib, tkinter and pygames.
Of course, tkinter is available out of the box with newer Python-distributions. But the resulting GUI is butt-ugly (I feel like using those UNIX-workstations in the first semester of computer science..) and you have not much influence on the layout. Since I am working for some while now in the background with PyQt (will be covered in one of the upcoming posts), I can say: good that I had a hands-on, but I will NOT use that.
matplotlib: high value in quick generation of plots of all kinds (bars, line-charts, pie-charts, ..). I’ve used it before and I guess this is the main earning from this learning-opportunity.
pygames: loading some graphics, adding a game-loop, reacting to user-input, all fine. But would require some additional effort for understanding. Maybe in the future.
Of course, a two-day workshop can’t provide you with credible knowledge and expertise for three frameworks. But having a teacher can ease the starting-pain and allows quick feedback in case something does not work. For me it was also a good opportunity to have some exchange with people and some learning-atmosphere. Also: since tkinter is so butt-ugly, I got further momentum continuing my PyQt-project.
People always talk always about shopping & happiness. Now I can understand :’) Just booked for the upcoming weeks for my education “Workshop: Deep Learning für Natural Language Processing (NLP)” && “iSAQB Certified Professional for Software Architecture – Foundation Level (CPSA-FL)” <3 #neverstoplearning
nota bene: and HSK1 for Mandarin will be tackled as well!
A note before the actual post: a week ago, while proofreading I’ve noticed that some of the following statements, which are meant truly neutral, could and would leave a stale aftertaste. That’s definitely not the intention; it’s more a snapshot of the current state (like for a chronicle). On the other hand: if I would censor it more, I can trash as well the whole post. Because ten thousands of texts were already written about the curent state of human society and the impact of Covid-19.
So, read it with a pinch of salt: we are lucky to be healthy and that we don’t suffer from more harsh conditions.
More than two months ago Sars-CoV-2 -induced infections scaled up in Germany and hit us without much preparation. Us includes me, my family, my workplace, society at all.
Challenge was to reconfigure a device, which has a Linux running on a TQ-board. Some files had to be adjusted.
Connect to the device:
(Then enter password.)
List amount of free space:
df -K .
Show target of symlink:
To give my (non-existant) skills for microcontrollers and reading out sensor-values a kick-start, I decided to connect today an DHT11 to the ESP32.
So, at first I realized I have no clue how a breadboard works, then that I need a calculator to determine the resistors properly, then “how to determine the correct pin-numbering”, then … it did not work, no matter what I tried. The C++-code was the least issue. Several sources provide examples, also with webserver (looks like everyone ~stole~ got inspired from each other ..), but the read-out always failed.
Luckily I had two (a bit more ‘expensive) DHT22 at hand and it worked like a charm!
Lessons learned: 0. accept that you know nothing 1. reading and experimenting is fun 2. seeing finally a presentable result is great <3
Humidity: 71.20 % Temperature: 24.40 °C
Temperature (internal): 31.67 C
The first month of 2019 already passed. And we passed it with flying colors!
But let’s have a look at 2018 – a year full of challenges and success: I’ve worked full-time, organized and participated in advanced courses for Python and in Requirements Engineering (officially: IREB Requirements Engineering Foundation Level-approved) and pursued a new employment as software engineer.
And I wrote some software in my spare-time, as you can see in the graph for the public github-repositories. The gaps in the commits can be explained with the birth of my daughter and the time where I acquired the new job and moved nearly 900 km across the country. Yay! Nice personal projects were and are Cullendula and the Daily Coding Challenges, which I solve mostly with fully Unit-tested Python (3).
More new, hands-on knowledge was gained in the area of CMake and Qt-charts.
Well – 2018 was great. Let me make 2019 greater! 💪
I am currently traveling back from my very first paid educational leave. Proper selection, arrangement and preparation lead to some awesome impressions: about the capabilities of Python and about the city of Detmold.
Daniel Warner lead us – an assembly of five inquisitive men in the age-range from 30 to 60 – along the details and
specialties of that programming language. I learned much, in detail:
- basic structures; list comprehension
- classes; objects; overrides; imports; representation; init-method
- dictionaries for caching results (memoisation)
- decorators (nice for for printing, caching and thread-safety)
- descriptors, properties and slots, kwargs
- (multi-)inheritance and its quirks
- recursive functions; functional programming
- threads, synchronisation, atomic access
I put all the exercises (full script with my own annotations) into a Git-repository right from the beginning and published it: https://github.com/marcelpetrick/Python_FortgeschrittenenSeminar/.
Which also makes a nice view of the github-history 🙂
Python was chosen by me by intent: I see and plan for ways to use it with artifical intelligence (TensorFlow-binding ..); microcontroller-programming (ESP can run MicroPython) and for the Raspberry (currently the tumblr-upload-script for the catcam is also Python); for daily data-manipulation-tasks which are currently done more or less on Bash or AutoIt or Batch – and then: write it once, run it both on Linux and Win).
This was a great choice! And I want to thank my wife for supporting these stays absent from home and my plan to achieve the wanted education 🙂 And I got a small certificate – but that’s just icing on the cake.
My plan as first real exercise is to re-implement the “find all islands in the given map”-programming challenge. This will be fun. Getting to know some specialties and what properties/slots mean in Python-context (compared to the Qt-ones) was nice. And the decorators are a really powerful way to add special functionality to methods without bloating them and without blocking the view to the busines logic.