Disqus - Latest Comments for jstrachan

Re: Continuous Delivery with Amazon EKS and Jenkins X

jstrachan — Tue, 26 Feb 2019 00:55:56 -0000

I'm so sorry to hear joining slack is currently unavailable. I guess we can use discuss.kubernetes.io until then - or you could raise an issue on github: https://github.com/jenkins-...

Re: How We (almost) Reinvented Jenkins X on Our Own

jstrachan — Mon, 28 Jan 2019 12:35:27 -0000

Good article! Note though you can still use Jenkins X with your classic hand-crafted Jenkinsfiles and pipeline libraries. You can then mix and match for the biggest win - use Jenkins X automation on new microservices you deploy on kubernetes and keep your existing hand rolled pipelines for other stuff.

Also by rolling it yourself you miss out on really useful features like:

* automated Promotion via GitOps: https://jenkins-x.io/about/...
* Preview Environments being created automatically on Pull Requests: https://jenkins-x.io/about/...
* develop inside the cloud with DevPods: https://jenkins-x.io/develo...

I know I can't live without Preview Environments now! They are awesome.

But yeah, if you are focussing on managing existing Jenkins pipelines as-is without taking advantage of kubernetes or GitOps then you need to weigh up if you want to roll things yourself or move to Jenkins X

Re: Continuous Delivery with Amazon EKS and Jenkins X

jstrachan — Mon, 19 Nov 2018 04:21:55 -0000

BTW the best way to get fast support is via the Slack channel: https://jenkins-x.io/commun...

I wonder if you could share more of the console output? I'm assuming something earlier in the output actually failed (e.g. maybe some resource could not be created due to your RBAC or IAM policies?).

Did you run `jx create cluster eks`?

Re: Comparing Tools That Offer Previews

jstrachan — Sat, 04 Aug 2018 02:19:43 -0000

There is also Jenkins X which is OSS and can be used on any kubernetes cluster - it automates CI/CD for releasing apps as helm charts and promoting them across your environments. It also supports preview environments on each pull request: https://jenkins-x.io/

Re: Deploy Monocular on OpenShift

jstrachan — Thu, 19 Apr 2018 23:49:04 -0000

Great blog post thanks! I've hit a similar issue of not being able to deploy monocular on OpenShift when it works on all other kubernetes clusters I try. https://github.com/jenkins-...

Though the setting mongodb.persistence.enabled=false looks kinda scary - doesn't that disable the user of the PersistentVolumeClaim - i.e. disabling persistence completely?

Re: Getting Started With A Local Kubernetes Environment

jstrachan — Tue, 31 May 2016 10:36:44 -0000

We struggled with the various options; particularly as we wanted to use CentOS and wanted to expose the Docker port so that we could easily do local docker builds from the host and reuse the built images directly inside kubernetes (to avoid delays caused by pushing to a registry), so we came up with this vagrant image: https://github.com/fabric8i...

It turns out its really easy to run a single node kubernetes cluster purely using docker too
https://github.com/fabric8i...

BTW here's now to get a nice console for running apps or a full CI / CD pipeline too:
http://fabric8.io/guide/get...

Re: Using NSEnter with Boot2Docker

jstrachan — Mon, 01 Sep 2014 12:28:52 -0000

awesome stuff!!! works a treat, thanks!

to get the bash completion thingy to work I did:

brew install bash-completion

then added the docker-enter_commands to /usr/local/etc/bash_completion.d

Re: Microservices and monoliths - is there a third way? - Zen Slapped

jstrachan — Thu, 15 May 2014 05:50:46 -0000

You're right that micro services isn't "new" (little in IT is ever really new since the 70s really ;) - its just a different approach.

While OSGi has been used for many years to implement modular services (and fabric8 started life working exclusively on OSGi & it still works great inside an OSGi container), most of the new 'micro services' based approaches and tools (e.g. DropWizard, Spring Boot, Vertx, Netflix & the ThoughtWorks approaches et al) are slightly different; its not so much about composing modular services inside a single JVM (the OSGi approach) but more about physically separating them into separate, stand alone JVMs for easier monitoring, management & incremental change.

While looking through OSGi glasses that might not seem a big deal - OSGi might equate to "micro services" to you - but from a devops perspective its quite different.

There are various ways to compose services inside a single JVM (flat classpath, DI, OSGi, servlets, JEE, JBoss Modules etc), IMHO the micro services approach is more about decoupling your JVMs/services. How the services inside a micro-service JVM are wired is a separate thing (and often micro services are so much smaller, the framework/app server is little less important). So I don't think anyone's really saying OSGi can't be used; any Java technologies which can load Java bytecode will do just fine really.

Though I would say, rather like JEE / EJB3, one of the drivers behind the micro services meme/movement/approach is on simplicity & developer productivity. Thats never a word even the biggest OSGi fan would ever use about OSGi compared to a flat class loader ;). IMHO the simplest, easiest, smallest & most productive classloader mechanism I can imagine today for writing micro services on is a flat classpath. Though YMMV. One things for sure; we're never all gonna agree on whats the one true way to make class loaders in Java ;)

Re: Parallel Universe

jstrachan — Thu, 15 May 2014 02:00:18 -0000

Thanks for your kind words! I guess my TL;DR; response is fabric8 is like a distributed version of Capsule for running more than one Java process on possibly more than one machine which may need to know where each process is on the network etc.

Yeah the fabric8 website needs some serious work to explain what it does; its kinda confusing and hard to grok from the site. The overview is probably the best thing we have so far, but I hope we can improve that soon! :)
http://fabric8.io/#/site/bo...
The video demo above may be the quickest way to grok fabric8 in the micro service context.

Basically fabric8 came from trying to make it easy to provision, configure and manage "Java stuff" in a number of processes across usually more than one machine; particularly REST + web services, Apache Camel integration flows and messaging applications using ActiveMQ / STOMP / AMQP / MQTT etc.

We designed fabric8 to have minimal requirements; its just a JVM you can start & its got everything you need & works on any OS with a JVM; but to be able to run multiple processes on multiple machines (and be able to help wiring things together). When you try run "java stuff" on EC2, on arbitrary compute nodes with jclouds, on OpenStack or on a PaaS like OpenShift or start using things like Docker; soon you tend to get dynamic IP addresses or ports and a significant amount of work is required to wire stuff together.

We started out supporting OSGi as the 'application server' for running Java stuff a few years ago when we started (integrating it into Fuse ESB then JBoss Fuse) then added the Java Container to support stand alone flat class path processes and are working hard on deep Tomcat, TomeE & Wildfly integration right now.

When it comes to class loaders and application servers we see a large spectrum from simple flat classpath, to servlets to JEE to OSGi; all have their strengths and weaknesses. Its a classic simplicity & productivity versus modularity/complexity trade off really. (e.g. OSGi is particularly great at reloading just, say, 1 bundle when new code is available without disturbing any other java code which can be really useful).

We see lots of folks in the field liking different parts of this spectrum for good reason; we kinda think 'the customer is always right' and want to let folks use whatever packaging/app server they wish for each service/application. Personally I love the simplicity and productivity of a flat classpath though ;)

In parallel we've added support to work nicely with different infrastructure (docker, cloud, IaaS, PaaS etc); so you can provision containers over jclouds, ssh, OpenShift and docker; but those are all optional; you can just use fabric8 on your laptop to run one or more JVMs using 'just java'. e.g. increasingly if you're using linux we see systemd as an ideal way to ensure your java processes keep running (which thankfully is the approach used by OpenShift too; so you're PaaS ready too if you go that route); but if you're on windows you can just miss out the systemd stuff.

I like Capsule though; it does one thing very well.

As an aside; the Capsule / Spring Boot approach of making an uberjar does make things easier to run ("java -jar FTW!") but one downside; it can take a little while to jar and unjar those jars in development (which in development they don't really add much value). Just putting jars (that change) into a lib folder and using 'java -cp "lib/*" $MAIN" does speed that up quite a bit; making the developer experience much slicker.

Re: Parallel Universe

jstrachan — Wed, 14 May 2014 11:00:01 -0000

Great article! I give jolokia a massive thumbs up too.

Another alternative to Capsule for provisioning stand alone Java processes from a jar (via maven dependency information) is fabric8: http://fabric8.io/

I did a little blog and video demo last week showing how to deploy a maven project to fabric8 and creating as many containers as required:
http://macstrac.blogspot.co...

here's the video:
https://vimeo.com/94501138

whats interesting is it works using separate processes locally on a machine; or can work on docker (where it dynamically creates a docker container image on the fly); plus we should add "Java Container" support to OpenShift shortly too.
http://fabric8.io/#/site/bo...

I like that each Java process can use any classpath it wishes; fabric8 doesn't add anything into the classpath or do any class loader tricks; so its nice and simple. Plus you get a nice out of the box hawtio console for each JVM you deploy (with fabric8 keeping track of all the containers and their status; where you can start and stop them via the UI or command line shell etc).

Then folks are free to use any framework they choose; dropwizard, spring, spring boot, vertx, dagger, CDI or whatnot; then whether folks are running stand alone Java processes, Tomcat, WildFly or Karaf containers; there's a single UI and CLI for provisioning and managing etc.

Re: Microservices and monoliths - is there a third way? - Zen Slapped

jstrachan — Wed, 14 May 2014 10:48:30 -0000

Its on my list of things to do to create a DropWizard demo of fabric8 provisioning. Its great stuff!

We've used Profiles in lots of different ways in the past (which support multiple inheritance and composition):
http://fabric8.io/#/site/bo...

e.g. we could have a profile, "activemq" for running, say, an ActiveMQ message broker on a typical box; then we could have a "activemq-bigbox" profile which inherits from "activemq" profile and just tweaks the thread pool size, connection count and memory buffer when run on a big box (with plenty of CPU and RAM).

So folks have often used profiles as a way of tagging groups of services (with hardware or geographical capabilities or metadata). As an aside, one nice thing is when using the RHQ monitoring software: http://rhq.jboss.org/ we've a plugin that groks fabric8's registry and mapping of containers (services) to profiles; so we can automatically aggregate metrics of all service instances of the same profile. (e.g. aggregate metrics for all message brokers globally; or aggregate by kind of hardware, by rack/data centre/country etc).

Sorry got side tracked there ;) what I meant to say was we could use profiles as 'tags' to indicate if a service was slow/fast or reliable or not, stateless or stateful etc. Then we could provision/manage services with different profiles/metadata differently. (e.g. we could use a different docker container or docker container configuration for stateful services to explicitly mount a shared block storage to backup the file system for stateful services - say).

e.g. maybe unreliable HTTP based services would get a Gateway provisioned in front of them to do automatic retries?

Having the perfect DevOps infrastructure for all kinds of services (micro or not) is complex and there are lots of trade offs but so far we've been really happy how fabric8 gets out of our way (using whatever container/application server/runtime/classpath/infrastructure we like on a per profile basis) then having a nice web UI and CLI for working with all this stuff; and we can then just perform some configuration or write some little bits of generic java code here or there (or create the odd custom docker image) to solve edge cases when we hit them.

Re: Microservices and monoliths - is there a third way? - Zen Slapped

jstrachan — Wed, 14 May 2014 09:22:18 -0000

Great post!

Incidentally I blogged last week and cut a little video demo showing how to provision maven jars with an executable main function using the open source project, fabric8 http://fabric8.io/ (with a management console for each JVM included too):
http://macstrac.blogspot.co...

you can either use stand alone processes; or if you enable the docker profile; then fabric8 creates docker images on the fly for each profile (based on the maven dependencies in your project) and uses docker to start/stop processes (and includes a nice web UI for docker too).

What I find particularly interesting about the Java Container approach in fabric8 is that there's not a single particular framework or container class added to the classpath - its completely up to the owners of a service to decide what its flat classpath is going to be; so folks can use vanilla Java, spring, CDI, vertx, dagger, dropwizard or whatever any versions they want; also its particularly cool that its so easy to view all the containers; their profiles and you get a nice web console for free to look inside each JVM.

If folks wanna use OSGi (or Tomcat / WildFly) its fairly easy to just change the profile and fabric8 deploys those containers instead; but in the case of Micro Services; avoiding the application server sounds like its going to be more common.

Whats interesting is that just starting/stopping a process / docker container isn't really enough; what we're finding is you need to do some wiring the micro services together (e.g. inject, say, system properties, configuration files or environment variables to say where, say, ActiveMQ message brokers are or where web service endpoints are etc). We can do this pretty easily with fabric8; whether its exporting host / ports / application specific metadata into the runtime registry; or using whats in the registry to inject environment variables / system properties / config file values..
http://fabric8.io/#/site/bo...

Another interesting approach - which is a bit more PaaS-ish or Docker-ish is to use localhost and fixed ports to communicate with protocols like AMQP / STOMP / MQTT or to talk to well defined services (memcached / cassandra or whatnot); then in this case you can also provision on the same machine a Gateway (like a local http proxy) that does the dynamic load balancing to where the micro services or infrastructure addresses & ports really are:
http://fabric8.io/#/site/bo...

Re: Writing a Camel Component – Camel-Static-Resource

jstrachan — Mon, 25 Nov 2013 11:47:39 -0000

Great article!

BTW if you also use Endpoint Annotations
http://camel.apache.org/end...

then you'll get a nicer auto-generated UI form in tools like hawtio - http://hawt.io/ - if you edit routes using this new component. You also get nice auto-generated static HTML too (similar to the auto-generated docs for maven plugins).

You can even implement the new EndpointCompleter to get nice auto-completion of the URI path in hawtio (and other tools)
http://camel.apache.org/end...

Re: The Best IT Events 2014: What Attendees Want and Exhibitors Expect, Plus ROI Considerations

jstrachan — Thu, 18 Jul 2013 06:50:49 -0000

BTW the Red Hat Summit / JUDCon also included CamelOne too
http://camelone.com/

which is the main conference for Apache Camel and open source integration technologies. Might be nice to mention it on the Red Hat Summit / JUDCon section.

Plus CamelOne has quite a lot of non Red Hat talks from various vendors & customers etc

Re: http://richardlog.com/post/21142113656

jstrachan — Mon, 16 Apr 2012 06:41:16 -0000

Note you could write the kotlin version as pretty much the same lines of code too BTW. e.g. line 13 could be:

!input.toLowerCase().split(" ").any{ !words.contains(it) }

i.e. scala and kotlin are about as concise.

Where sometimes Scala can be more concise, its usually at the cost of readability and easy understanding; though in general application code they appear to be about as concise as each other

Re: Continuous Integration and Feature Branches

jstrachan — Wed, 20 Apr 2011 08:23:11 -0000

FWIW we use a little bit of scala code to auto-generate most of our CI build configurations; so its really easy to add new projects, branches, mvn profiles and so forth using a simple little DSL describing the build configurations...

https://github.com/fusesour...

Re: Maven artifacts need to be more discoverable

jstrachan — Mon, 17 May 2010 04:00:42 -0000

Am liking the use of URLs instead of dependency group/artifact/version tuples (particularly as often projects change their group/artifact ids or moves stuff to different repos which causes pain in poms).

Then you could just run (on your local machine or on your WAN/LAN for your team) a regular caching http proxy to speed up mvn builds.

It is nice having a global mirrored repo as a cache in case sampullara.com is down for example - but central repos could just rsync known repos and act as a backup http proxy cache.

It'd be nice to be able to do imports using URIs too in code...

import http://cli-parser.sampullar...

and the JVM could auto-download stuff on the fly...

Re: The ideal web application templating system

jstrachan — Fri, 07 May 2010 12:30:29 -0000

I'm not sure there is a single ideal template language; it kinda depends on your requirements.

e.g.

(i) does the designer just use mock data inside a local HTML file, then you inject values into it on the server,
(ii) do you use mustache style tags {{foo}} or {{#bar}} inside the HTML which look a bit icky to the designer but at least there's no logic - and you can reuse templates on the client side too
(iii) or do you have the templates owned by scala/ruby/java/groovy hackers so they want something DRY (like Haml)
(iv) or use a template language the developers already know well (SSP/JSP/Erb/Velocity etc).

They all have strengths and weaknesses depending on the project and make up of your team. FWIW we've got (iii) and (iv) in Scalate already and we're working on (i) and (ii) in master right now.

http://scalate.fusesource.org/

Irrespective of (i), (ii), (iii) or (iv) you get a nice error console which highlights to the developer which lines/expressions have errors when mistakes occur in development mode. The Scala parser combinators for all the above templates helps us make really good error messages if there are any mistakes in the templates (very common!). Plus the API and layouts are polymorphic so you can mix and match the template engines based on requirements.

e.g. the designer may own some templates (i) or you might use mustache style (ii) or use scaml for developer-owned templates and (iv) for emails etc.

I've ran out of time this week - but I hope to document next week how you can use either mustache style or jquery style templates together or separately, depending on if you want a plain HTML without mustaches or not for your designer...

Re: Why Apache ActiveMQ isn’t good for .NET developers?

jstrachan — Wed, 09 Sep 2009 04:00:41 -0000

@Alessios - agreed. Thats the 'messaging' way of doing things; which is based on asynchronous message passing and avoids locking yet works in a consistent way despite high concurrency rather than doing non-transactional RPCs (querying queue depths) etc.

i.e. to use messaging well you don't treat the messaging system as a database to query

Re: Why Apache ActiveMQ isn’t good for .NET developers?

jstrachan — Mon, 07 Sep 2009 10:33:56 -0000

NMS should be able to support queue browsing - see http://issues.apache.org/ac...

FWIW you can use the web console to view the queues and browse messages. We're in the process of creating a fully RESTful access to ActiveMQ from any language via http://restmq.fusesource.org/ so any language capable of doing a few HTTP GET requests can find out details on a destination, browse its messages, DELETE them or POST new messages etc.

In the meantime I've raised this issue which would make it pretty trivial to query the queue size, by just sending a message to a pseudo destination of ActiveMQ.Statistic.Queues.MyQueue and getting back a Map message of the current size & enqueue/dequeue rates etc.

See https://issues.apache.org/a...