simple webscraper for last.fm with BeautifulSoup
tl;dr:
Simple webscraper with Python and BeautifulSoup for one user’s favorite tracks (‘loved songs’) at last.fm.
Repository: github
full text:
Looks like last.fm is shutting down its services (one feature at at time, lol). They started this process more or less ten years ago.
I’ve realized that I would miss my curated list of favorite tracks and I am also very bad at remembering, so … let’s automate the process of grabbing that information from their public page. I know, they offer a REST API, but I wanted to use once BS4.
Since I had somehow two free hours fourteen days ago, I went full-speed to some tutorials, played with the get-requests and how to parse. And then spent the last minutes parsing the received “artist+track”-string into something usable. Alltogether four hours were spent and I am amazed by the result. Of course, by leveraging three quite powerful libraries (beautifulsoup4, requests, lxml) and skipping TDD (;) I’ve reached the goal quite fast. And since the script works (my 1500 loved songs are scraped in less than 60 seconds), I will also not spend additional effort to make it “pretty”.
“[Full Day Workshop] Kubeflow + BERT + GPU + TensorFlow + Keras + SageMaker”
I’ve just spent the last eight hours attending a workshop about #SageMaker, #AutoPilot, #BERT, #Athena, #TensorFlow, #Spark, [..] and I am feeling a bit light-headed.
Of course, the talk and guidance given by @AntjeBarth and @ChrisFregly was really well prepared, but if you’re just a ML-beginner (like me) and if then over 9000 of new technologies drop, you have to work hard to follow the fast paced event.
Of course, I started my ML-journey in the summer of 2019, but it was more focussed on image-processing, not #NLP. I worked before with #Python, #Jupyter notebooks, TensorFlow and #Keras, but that whole SageMaker-thing was new to me.
And I see the potential: instead of running the stuff locally, you prepare, prototype and run your ML-app inside Amazon’s infrastructure. And that AutoPilot, which helps to quickstart the prototyping by trying several preprocessing-steps and models for you on your data, looks promising. Will definitely give it a second look.
Notes can be found at: https://github.com/marcelpetrick/KubeFlow_BERT_GPU_TensorFlow_Keras_SageMaker_Workshop (need lots of polishing, as always)
Crazy times we live in! And I am thankful for this block of time on a weekend 🙏
current workplace IV
* development with C++ 17; Qt 5.12; CMake 3.16; Gradle as build-configurator and extool-packages for external dependencies
* product built for Win (10) and macOS (10.15); one graphical UI and two SDK (C and C#)
* TeamCity for CI; Gerrit (code-review); JIRA, Confluence, Rainbow & Outlook (communication)
* team-size fluctuates: 5 to 11 developers, PO, PM, ScrumMaster, 3 testers
* external stakeholders inside the company; medium company size; global distribution
* development happens in sprints of 4 weeks; DoR (Definition of ready) and DoD (Definition of done) are enforced for tickets; root-cause-analysis is done for bugs; capacity is evaluted before each sprint
* releases are created roundabout four times per year
* additionally to my role as SW-engineer I did the job of the “release responsible” in the last year (documentation preparation, test-management, release-creation, RC-handling)
GammaRay – another debugger for Qt
Small OpenSource-tool from KDAB which can be used to alter the runtime-state of Qt-based programs. I’ve just used it (so far) with Qt for C++ and wonder if it would work with PyQt as well (TODO).
Changed parameters, layouts, styles, sizes, values are immediately applied and helpful to fine-tune especially layouts. It is some sophisticate debugger. Of course, with gdb you can alter values as well, but this Qt-aware tool offers for certain types (or layouts) just valid entries and ranges!
Code can be found here: https://github.com/KDAB/GammaRay/wiki/Getting-GammaRay
Some more documentation can be found here: https://www.kdab.com/development-resources/qt-tools/gammaray/
———————————————
Quick-start guide:
First of all: the Qt-version of the GammaRay and of the UUT (unit under test; the program) should match. If not, then runtime-issues up to crash are possible 🙂
As said: almost everything can be influenced, so for a limited task, I’ll show how to alter some style of ‘Cullendula’. Let’s set for it nice blue outlines for each widget!
0. start Cullendula
1. start GammaRay (bin/gammaray.exe for Win)
2. attach GammaRay to Cullendula via selecting it [1] and then switching (if not done automatically) to the proper build-version & -type of Qt [2]
———————————————
3. in the next window search for “stylesheet” (best done via the search-box)
———————————————
4. Now insert for styleSheet: “QWidget { border : 2px solid blue}” (without quotation marks!) and confirm with . The changes are applied immediately.
The same workflow can be applied almost everywhere to manipulate other properties .. good luck.
It’s a quite useful tool for layout-debugging. Also the measurement-tool can help to fine-tune margins .. or discuss color-values with stakeholders ..
Qt6: preview is available and usable
After reading this blog-entry https://www.qt.io/blog/qt-6.0-feature-freeze-milestone-reached I wanted to give it a try and see how much effort it would make to transition from Qt 5 (5.12 or 5.15) to 6.
Hands on-approach: installed Qt6 via the maintenance tool (1 hr and 39 GiByte later ..), took my Cullendula-app and added a new target within QtCreator 4.12.
First clean build with MSVC2019_64bit gave a weird warning about
qtmaind.lib(qtmain_win.cpp.obj) : warning LNK4099: PDB 'WinMain.pdb' was not found with 'qtmaind.lib(qtmain_win.cpp.obj)' or at 'C:\Users\husband-boy\Desktop\coding\Cullendula\debug\WinMain.pdb'; linking object as if no debug info
where I still found no solution. But it works. Not a single line of code had to be changed 🙂 If anyone ever transitioned from 3 to 4 (big ugh), or 4 to 5 (small ugh), you know the pain.
Bravo, I am positively impressed. Nothing worse than stagnation.
PyQt: GroundSpace
Over the past weeks I’ve worked on a small project to combine the best of the Qt and Python domains. It was time to put both together. I knew about the PyQt- (Riverbank) and PySide- (Qt) bindings for years, but never really dipped my feet into those water. It was time to fix this.
GroundSpace (wordplay) is a small tool to fill your hard-disk (SSD ..) with arbitrary content. To test the speed of writing and to create big chonks of data.
What was learnt?
* creating an ui-file with QtDesigner (jk, I knew this) and how to pre-compile it for PyQt-usage
* loading that uic-file and creating connections
* progress-callback
* how evil the ‘eval()’ function in Python is
Next stop: I want a proper web-scraper in Python.