Disqus - Latest Comments for otobrglez

Re: Attending a heckathon with Scala and Akka (Streams) on Microsoft Azure

Oto Brglez — Wed, 07 Oct 2020 20:04:09 -0000

Hey Ula!

Thank you for your kind comment. I think that this is the 12th hackathon that I’ve attended... I believe that I’ve attended em’ in 4 different countries now. 😅 Every event like this is unique, and every event has its ups and downs. The common theme in all of them is that it’s not really about winning; its more about connections and people you meet. I meet people that I now consider to be dear friends,... founders, engineers, designers,… magical! 🧙‍♂️

For me, the nice challenge is to learn more about different domains, quickly. As I see that to be one of the critical advantages to modern architect/software engineer - being able to adapt and to get the domain, problem and find potential solutions. Beautiful!

For this particular event, I think that I’ve found a few exciting angles that will shape modern enterprise/business. And it goes something like this;

1. Business needs to transition from “data is oil” mindset to the next level, where we need to put that oil to use. And that is very, very hard; as the data is stored in silos/systems, gated by domain experts. So the challenge is to make a unified layer on top of all data and enable experts to collaborate…

2. Although the business has domain experts, that doesn’t mean that the experts are also automatically experts in processing or knowledge extraction. Example: Meteorologist is an expert in weather, possibly some statistics, … but does he have the skills to combine all the data sources and build or use a warehouse? Should he? Or should we give him a layer that he can just use,... instantly?

3. I believe that we are now entering the next stage of “cloud phase”; where tools or rather abstractions on top of clouds, will open new and very unique opportunities and solutions. Think about how hard it was to build something like “water archive” 10, 15, 20+ years ago,... you needed servers, sys-admins, engineers, dba-s, good knows what… Now its 2 clicks and an “Azure Function”... Just focus on business and let the Azure handle the rest. Beautiful.

4. Historical data and real-time data is “just data” that can be combined to provide new insights and be used to build new products that were impossible before.

5. Weather data or understanding of the data is vital, perhaps even more than ever before. We need to evolve tools and technology to get ahead of the curve.

I want to thank you and Microsoft team for supporting this initiative and giving attendees, and me a chance to play on some of MS technology!

Fun! 🚀

Re: Attending a heckathon with Scala and Akka (Streams) on Microsoft Azure

Oto Brglez — Wed, 07 Oct 2020 18:49:45 -0000

Hey, Jernej from Google 👋!

Thanks for your excellent question! There are several reasons why adding a broker like Kafka in the middle or should I rather say the heart of a modern system. It goes something like this,...

1. We are capturing water data from a “live” feed, so the assumption is that unless you capture this data then and there it will be lost and it impossible to retrieve it ever again. So, if any component in the chain that we built for ETL fails, we’ll lose the data (networking problems, deployments, service overloading etc,...) Although your assumption is correct I could just save the data to PG or even some NoSQL system… but what if my parsing was wrong? What if my mapping was wrong? What if my schema was off? I again lose the data. But with Kafka in place; I can just go “back-in-time” and roll the events back from a particular point (offset), change “things” and let the flow do the rest. Fix the post-processing, and proceed with consumption,... Upgrade PG, proceed with consumption…. Change PG with Cassandra, proceed,...

2. The common misconception about modern Kafka is that it is “just a broker” or just temporary “pub/sub” mechanism. It is way more than that; especially with tools like ksqlDB. You need to think about Kafka in a more persistent way. Think about this like this - Kafka is your also your storage - persistent, distributed and resilient.

3. My app handles essential interaction to targeted Web Service, a low-level transformation from XML / JSON to targeted data structure (for Kafka Avro would be preferred so that Schema Registry can be used) and pushing the data to Kafka (or Event Hub in the case above). And that is it. The “client” is dumb. The only smart thing about my client is that it can back-off and that it can survive fundamental problems that systems that handle reactive manifesto (™) do. The protocol is also pretty trivial. (no f* with PgBouncers, no f* with partitioning, no f* with anything PG specific at this stage). Dumb clients are also traditionally trivial to deploy and easy to scale. And this translates into fast release cycles,...

4. ksqlDB is then the place where transformation happens,... its a place where you define target topic and behaviour of them (replication, etc…) This is also now a place where data goes from unstructured to structured,... It is now also a place where I have full control over the processing. The other magical thing about this thing is that you can JOIN different streams (made from Kafka topics) together. So in my case, I could have a topic of live measurements, a topic of all stations with location and a topic for historical measurements, these could then be joined together and streamed into another topic that could then either be materialized or feed into PG or some other analytical storage.

5. I could also use MQTT, with MQTT broker…. Or MQTT with Kafka’s MQTT connector this would then be perhaps even simpler and lightweight implementation,... But then some of the transformation things that I’ve exposed above would be harder to do and my experimentation would be more challenging.

So, summary. Although it looks like overkill from high-level, it simplifies so many things and makes experimentation with full piping, collection and transformation easier. It also makes deployment, scaling and debugging a breeze.

I hope that this sheds light into my thinking about this and I hope that we’ll dig deeper into debate next time when you are in town. 🍻

✌️

Re: Reverse engineering an Android Appplication

Oto Brglez — Tue, 01 Sep 2020 10:50:17 -0000

That's a good subject. Thinking about doing a followup article where I would explore ways to inject code into apps and repackage them back together. That would be something. 🤔

Re: Reverse engineering an Android Appplication

Oto Brglez — Fri, 07 Aug 2020 07:22:10 -0000

Hey @disqus_5pWHbZj6PG!

Until the way that Android apps are build, distributed and ran changes. It will very likely always exist a way to break them apart - to an extent - and figure out how they work. You could make that procedure harder and not worth the effort...

You could for example run some obfuscation tools in your compilation procedure that would make re-engineering harder. Check tools like ProGuard or DexGuard.

Or perhaps you could also write few parts of your app with an assembler or something similar that would be perhaps bit harder to re-engineer. But that then opens another set of problems altogether.

Cheers!

Re: Reverse engineering an Android Appplication

Oto Brglez — Fri, 31 Jul 2020 08:20:46 -0000

Hey guys! I've introduced Disqus commenting system to epic.blog site. Hopefully to capture some constructive comments and debate. Welcome!

Re: Redis 6.0.0 GA is out! -

Oto Brglez — Thu, 30 Apr 2020 16:33:35 -0000

I love Redis. Good job guys! 👏 Thank you!

Re: Learning Akka Streams - cjwebb.github.io

Oto Brglez — Tue, 31 Oct 2017 20:39:22 -0000

Thanks for writing this!

Re: The making of the concrete fibonacci clock

Oto Brglez — Mon, 07 Nov 2016 14:49:45 -0000

Wow. Very impressive and love the idea of the concrete! Congratulations! You should be proud! 👏

Re: Getting Started with RxJS » Miguel Mota | Web Developer

Oto Brglez — Tue, 02 Aug 2016 05:52:40 -0000

Thank you for writing this piece. It is informative, clear and easy to understand. Cheers!

Re: WordPress Developer’s Intro To Docker

Oto Brglez — Tue, 10 May 2016 17:48:12 -0000

Yes it is. As much as whole Docker. ^^

Re: Creating a Microservice? Answer these 10 Questions First

Oto Brglez — Wed, 03 Feb 2016 04:47:37 -0000

Very very nice post! Just please don't forget "local" developers. How services can be developed locally - if needed in isolation from the rest of the system... I believe that that thas is also important.

Re: From Sinatra to Amazon Simple Queue Service (SQS)

Oto Brglez — Thu, 07 Jan 2016 06:08:28 -0000

Yeah. There is no Sidekiq here. Just plain Ruby and aws-sdk.

Re: Using Docker for Rails Development

Oto Brglez — Wed, 09 Dec 2015 10:25:47 -0000

Nice article! You should check "onbuild" flags / tags. Much smaller images, faster work...

- rails docker onbuild -> https://github.com/docker-l...
- more details here -> https://hub.docker.com/_/ra...

Re: Izkušnje uporabnikov prenovljenega portala

Oto Brglez — Fri, 27 Nov 2015 04:54:54 -0000

Pogrešam kritično razpravo o tem zmazku! Stran je
1. nevarna - prijave / registracije (očitno plain text shranjevanje gesel)
2. izjemno slabo skodirana (neupoštevanje minimalnih standardov)
3. več kot očitno predraga (izhajam iz izbire tehnologije in izbranega izvajalca)
4. slabo testirana
5. nepregledna
6. vsebine so izjemno slabo in preveč birokratsko napisane. NISO vsi ljudje magistri prava.
7. manjkajo osnovne funkcionalnosti (ni obvestila o piškotih, kljub temu da se uporabnikom sledi via piwik)
8. Nejasni in nelogični delovni tokovi. (kam oziroma komu gre komentar, ki ga objavim na sebino? Kdo je odgovoren da odgovori? Kdaj bo odgovoril? Kaj so drugi ljudje vprašali?)

Napisati da je v "beta" je prelaganje odgovornosti in kar kliče po aneksih. Stroka že vrsto let odsvetuje uporabo oznake "beta", ker le ta vzbuja nezaupanje v izdelek in kaže na nezrelost izvajalca in njegovo nezmožnost doseganja dogovorjenih rokov.

Upam, in želim, da računsko sodišče oziroma KPK podrobno pogleda kako se je porabil denar za tole.

LP!

Re: Maturo razumemo kot generator malih lopovov

Oto Brglez — Wed, 20 May 2015 05:40:49 -0000

Hvala @Matej Praprotnik ! U r0x. :)

Re: Maturo razumemo kot generator malih lopovov

Oto Brglez — Wed, 20 May 2015 04:31:42 -0000

Sem sam jz al je tale posnetek zjeban? LP!

Re: How to use environment variables in your Angular application Cómo utilizar variables de entorno en su aplicación Angular

Oto Brglez — Fri, 10 Apr 2015 10:14:19 -0000

Hey man! I spent hours and hours debugging this thing. :) There is error in your code.
dest should be "'app/scripts/config.js'" and NOT "<%= yeoman.dist %>/scripts/config.js".

Re: Using a game engine to draw a 3D graph crossplatform

Oto Brglez — Tue, 28 Oct 2014 13:31:03 -0000

Hey Anže!

Nice article & code!

- Checkout this video for some more inspiration
~> http://www.youtube.com/watc...

- I also made a little visualisation with... well.. lets say it different approach.
~> https://vimeo.com/96479940
~> https://github.com/otobrgle...

Keep hacking!

Re: Aloo - GIST for data / RailsRumble 2014

Oto Brglez — Sat, 25 Oct 2014 04:04:39 -0000

Hey @Brandon Hansen! Happy to hear that it worked for you? Please share the link to your submission so that we can check it out! Cheers!

Re: 2014 Rails Rumble

Oto Brglez — Wed, 22 Oct 2014 09:01:59 -0000

I did! =) Please checkout Aloo - my submission. http://railsrumble.com/entr...

Re: 2014 Rails Rumble

Oto Brglez — Tue, 21 Oct 2014 13:45:06 -0000

Trolololo all the way. \m/

Re: 2014 Rails Rumble

Oto Brglez — Tue, 21 Oct 2014 13:42:08 -0000

Love the interface, love the product, you guys did a nice job with this one!

Re: 2014 Rails Rumble

Oto Brglez — Sun, 19 Oct 2014 18:07:39 -0000

I would appreciate some comments about Aloo. :) Thanks to everybody!

Re: Realtime personalization and recommendation with stream mining

Oto Brglez — Tue, 30 Sep 2014 07:58:37 -0000

Thanks for sharing! Keep being awesome!

Re: How to Design a Great Web API | Toptal

Oto Brglez — Wed, 11 Jun 2014 03:20:54 -0000

Couple of months ago I came across a free book from Brian Mulloy from apigee titled "Web API Design - Crafting Interfaces that Developers Love". He describes what I think are the best good and bad patterns when designing / building APIs. Versioning, Error handling, formats, authentication, ect.. Book covers everything what I think that good API developer should know and like said its free. http://info.apigee.com/Port... P.s.: Its language agnostic, so don't be afraid. :)