Cloud analytics – Using cloud based services to analyse big data
Last week I presented on cloud analytics at the Massey ecentre’s ‘Into the Cloud’ event. The slides are now available on Slideshare.
Imagine if your neighbourhood was one where criminals blatantly plied their trade down your street, banging on your door at all hours of the day and night and extorting money from you, breaking into houses and taking credit cards, stealing your money to pay for terrorism and other inhuman activities. Imagine that that some of the products you buy compounded the problem because the vendors published all your personal information without checking with you first. Worse, if you called the police, they wouldn’t be able to do anything, because the criminals live in a different neighbourhood where the local law enforcement don’t give a damn. You’d probably like to move, right?
Unfortunately we all live in this neighbourhood, it’s called cyberspace. Every day I am subjected to phishing attacks. Every day I am compelled to use software that is about as secure as a broken padlock. Facebook makes public things that I am sure I never agreed to (but then who can read those pages of tiny print with all the get out clauses). Flickr is determined to give all my photos away unless I constantly tell it not to. More and more of my intellectual property is somewhere ‘in the cloud’ where levels of security are an unknown and there are no guarantees that i can access my own work when I need it.
Maybe we should all wake up a little and realise that we live in a dangerous neighbourhood. Apparently 1.9 million people who had their password stolen from Adobe were using the password 123456.
http://www.bbc.co.uk/news/technology-24821528
Not, of course, that a ‘stronger’ password would have done any of these people any good if it was stolen anyway. In the end, the more you live on line, the more of your life you have given away for good.
‘Back To The Future’ was recently on TV again. According to the movie, I should now be driving a fusion powered flying car. Oh well. In another example of how things change more slowly than we may think, I just attended a presentation on Embarcadero’s latest version of RAD Studio, presented by David Intersimone (aka David I). Now David I has been around a long time – he (indirectly) taught me C++. Over 20 years ago I learned to program in C++ using one of Robert Lafore’s excellent C++ books, which came with an interactive computer-based tutorial. Along with this I watched a set of Borland videos, ably presented by David I. I also had access to the Borland C++ compiler so could try things out as I went along. So anyway, there I was at the beginning of 1990s having an interactive multimedia self-paced hands-on learning experience. I must admit that I’m finding it hard to see the vast strides we have made since, apart from the fact that we now have the web, which we didn’t back then, in fact I didn’t even have email. So the delivery channels have changed but I wonder what else we’ve achieved? The latest thing in self-study is, of course, the MOOC, and I have recently signed up for Waikato University’s new MOOC on data mining. If those guys can teach me half as much as David I and Robert Lafore did in the last century then I will very happy.
The Hotel California of Cyberspace
A great article from the New Zealand Listener web site about how the British science fiction writer J.G.Ballard predicted social media back in 1977. He didn’t make it sound very appealing. we are all just prisoners here, of our own device.
The future of the Performance Based Research Fund
An interesting article on the Tertiary Education Union web site about the future of PBRF, and comments made by Jonathan Boston, one of its original architects. Among other negative effects of continuing with a 4th round of PBRF, Boston warns of a ‘research treadmill that deadens the soul rather than inspiring the mind’.
As editor of an international journal I have to manage submitted papers, authors, reviewers, reviews, editorial decisions, final materials and publication schedules. Doing this manually is very tedious so I was pleased when the publisher announced a new on-line submission system to help with these tasks. Unfortunately, it is a very poor system. As a reflection on the process of project management, and what can go wrong, I believe the following issues are the most problematic:
1. Not involving the appropriate stakeholders early in the project.
2. Not making it clear on deployment if the system was intended as a beta test version or a full release.
3. Not providing a clear process or contact person for reporting bugs.
4. Not responding to bug reports
5. Creating a system around a rigid business process as seen from the management perspective (from some theory about how journals work rather than actual experience)
6. Not testing that outputs are correct (for example an old version of a paper appears as the current version in some cases)
7. Sending messages from the system using a generic source address so messages sent from the system disappear into spam folders and are never seen again.
Like many systems, this one works partially, and can be used with a few workarounds that bypass its more bizarre features, but it is a shame that a system that could have been so useful is more often than not a source of frustration.
Well, OK, gross exaggeration, but not entirely untrue. I regularly receive those email offers from ‘daily deal’ web sites, where you get 24 hours to pick up a deal. I must confess some of these have been a bit dodgy. for example, what possessed me to buy a $65 video camera? Anyway, a few weeks ago there was an offer for a mole map and skin check, so I decided to take it up. Turned out I had a mole on my back that I couldn’t see that had to be urgently removed. The biopsy showed it wasn’t yet a melanoma, but had I not been inspired to have the skin check by my web-based daily deal it might have been a different story. Around the same time, I took up an offer to update my will. Luckily the two turned out not to be connected. Anyway, the purpose of this post is to encourage everyone to get regular skin checks. You never know, it might save your life.
I had an interesting email exchange with one of our extramural students who works as a Business Analyst. He was recently told by a developer that class diagram modelling should only be done by developers, not by the BA. However, I imagine the developer’s definition of ‘class diagram’ was the kind of detailed design level class diagram that developers might use. The type of class diagram we use in analysis is the domain model, which uses a subset of the notation of a class diagram. The role of the domain model in the analysis phase is to clarify business rules such as, for example:
These types of rules can be expressed in text, but I personally think they are much better expressed in a domain model. This would only include business concepts and their relationships, with some key attributes, and possibly some core operations. Here is a very simple example:
This part of a domain model expresses that a policy holder must hold one or more policies, that policies have one (and only one) policy holder, that zero or more claims can be made against a single policy, that claims have a reference number, that policies have a policy number, and that policyholders have a customer ID.
To me the diagram is simpler, clearer and more explicit than the text version of these 6 core business rules. The question is, I guess, whether your customer feels the same way. I would suggest that the effort involved in learning that a rectangle represents an important business concept, and that lines show important relationships between these concepts and their multiplicity, would be trivial compared to the benefits of clearer, unambiguous definition of business rules. I’d be interested to hear from any BAs about whether they do or don’t use domain models.
With new classes about to start at Massey University in Systems Analysis and Design, and System Management and Testing, I could not have wished for a more topical local software systems disaster than the the New Zealand Novopay debacle. What a great case study in how not to procure, deliver, test or manage software! There can be few people in New Zealand who haven’t been affected by this deeply flawed system.