Archives for Kyle Drumm

Creatively move Mac OS X 10.6.8 Wiki to SharePoint Online

Isn’t it frustrating when it’s nearly impossible to find information on moving data to new systems? This is one reason I shy away from recommending data systems that are not business-class… such as the Wiki server feature found in OS X Snow Leopard. I wanted to share this post due to my frustration of not being able to find an easy migration path away from this Wiki software and onto something more business-friendly. Hopefully it can help someone in a similar situation.

I’ve been working a lot lately in Microsoft Flow – an automation tool that works great for workflows and connecting different software packages to make them work together. Because Flow is designed for repetitive tasks, it seemed like it could be a candidate for this function even though you wouldn’t normally think of it for this purpose.

Here’s a little background on the native state of the Wiki, and what we’re dealing with:

  • The raw files are stored on the Mac server in  /Applications/Server.app/Contents/ServerRoot/usr/share/collabd
  • Each subfolder.page is basically an individual post.
  • The files in each of these folders that you’ll want to work with are the .plist files. You’d think that you’d want the .html files, but Apple doesn’t make it that easy for you. Looking at the .plist in a text editor we see that the .plist is where almost all of the post detail is stored in an HTML-like format.
  • If the post included attachments such as files or images, they should also reside in this folder.

Let me start off with a couple of caveats before I go into the technical detail of how this will work. First, this is a basic migration solution. There is no native compatibility between the OS X Wiki and, well, pretty much anything else. This process at least allows us to grab some of the more useful information from the Wiki and take it to a system which will hopefully be a more future-proofed home; in this case, SharePoint Online. But because of this, we’re going to lose a lot of the features that were used on the Mac Wiki: user comments, inline pictures, and file attachments are some of the things that will not carryover (or at least nicely). If the Wiki was used as a knowledge base of sorts, which my example was, then this may be OK. The frequently-accessed KB articles can be easily mended by hand and the others can be handled by attrition.

OK, lets get to it. We’re going to start with a blank Flow. Now right off the bat you’re going to have some choices to make; the first of which being how to get Flow access to the .plist files to begin with. I chose to accomplish this with a OneDrive folder. This worked really well for a couple of reasons:

  • Flow can access OneDrive for Business with a native connector, and you can trigger your Flow when it sees new files in the specified OneDrive folder. This is just straight convenience.
  • If you have Flow you should have OneDrive, so why not?

Whatever you choose, one more catch with Flow that hung me up for a second was the file format. It didn’t want to try to read the data from a .plist file. So before you bring them into Flow, change the file extension to something less Apple-ish such as .txt or .html.

Once you have your file import method sorted out then we need to start pulling the data out of the .plist’s. I first identified the data fields that were most important to me that I needed to make sure pulled over into our new Wiki. For me, these were:

  1. Title
  2. Author
  3. Created Date
  4. Body
  5. UID (I’ll explain this one more later)

It’s important to understand the layout of the .plist files. Within the .plist each of the fields we want is encaspulated in <key></key> tags. Then, the content that corresponds to that field will immediately follow it within <string></string> tags. Within Flow we can use this to our advantage to use the particular key tags to determine where in the file to pull content from.

In Flow, I started with initializing a string variable for each chunk of content that I wanted to end up with:

For the Title, I used substring() to extract the data between certain points starting at the phrase “>title<” as shown below:

substring(triggerBody(),add(lastIndexOf(triggerBody(),’>title<‘),22),sub(sub(lastIndexOf(triggerBody(),’>tombstoned<‘),15),Add(lastIndexOf(triggerBody(),’>title<‘),22)))

Basically all we’re doing here is calculating the position of the start of the data (we know where it starts because it begins with <key>title</key> and is immediately followed by the next content tag <key>tombstoned</key>. Notice in my expression above that I chose to use >title< and >tombstoned< rather than <key>title</key> and <key>tombstoned</key>. This is because I found that the substring() function in Flow did not seem to like the expression if it was built with these full tags, something with the extra special characters made it throw an error basically stating the there was zero-length content.

Luckily for us the date is in a format that is easy for Flow to consume. Since it does come over in UTC though, after we grab the content using substring() we send it to the ‘Convert Time Zone’ action to get it into our local time zone:

The actual body of the Wiki post posed a new issue for us; look closely and you’ll see that it is typical HTML, but uses character entities for the greater than/less than symbols: &lt; for < and &gt; for >. If you want to leave them you can, however for my purpose I took them out using a couple of quick expressions using the replace() function:

replace(variables(‘varHTMLBody’),’&lt;’,'<‘)

Now that we have our data, it’s time to start inserting it into HTML that we can usefully take wherever we want. I accomplished this by using a number of Compose actions that brought the content into the correct places between HTML tags:

… and further brought individual pieces together with concat(variables(‘strHeaderHTML1’),variables(‘strHTMLBeforeTitle’))

Take note in the image above that I included the original page’s meta data as part of the HTML header. What this does for us is allow us to very quickly match one of our new Wiki pages with the old folder from OS X. This gives us the ability to go back and grab attachments, images, or the raw files again if needed.

Now that we have our HTML content all brought together, in order to build the file itself I once again turned to OneDrive to build a brand new file with the HTML content. I also saved my files as .aspx since I wanted to ultimately bring them into SharePoint:

We’re technically done with the conversion at this point; but why not use Flow to also get the converted file to its final destination? I mentioned earlier that, for me, this was SharePoint Online. Good thing we used Flow, we can do this part automagically too:
We can do a little better than this, though. I brought the files into a Wiki page of SharePoint, so I’d really like them to be as useful as possible to the staff who will be using these pages. Sadly I still can’t make it work to where they could be natively-edited in the new format and in the new Wiki, but the best I can do is give my users a table of contents so they can at least get to the pages quickly and easily. To help me accomplish this I used Flow to build the hyperlinks that I’d need for this and dump them into an Excel spreadsheet that I can copy from. I can build the table quickly and easily and then just copy and paste it into the Wiki Home.aspx page so that at least it looks nice and native for our users:
In the end, we have something that is at least useable. This is quite a hack to automate the migration of the OS X Wiki into something else, but I honestly did not have any luck finding 3rd-party software to do it for me. If you know of something, please let me know. Good luck!

Traditional vs. Next-Gen

I had an interesting conversation several days ago with a network admin who was looking into making changes to the network at his company’s main office. This office housed around 100 folks and was fairly straight-forward with technology needs. They had a handful of VLANs for different departments and functions.

What I liked about this setup was the fact that the VLANs were all trunked through to the pair of high-performance, high-availability firewalls at the office that were also the site’s L3 routers. In this way they were able to apply security filtering (AV/IPS/App control) to all inter-VLAN connections rather than leaving this protection at the internet border only. The network admin that I was conversing with wanted to break off this routing, though, so that all VLANs terminated at a dedicated router and the firewalls would only be used as the border gateway.

This is the traditional Cisco way of thinking, and functionally it works. It works great! I have a background in Cisco networking so I understand this very well, and I also realize that different size networks will have different needs – not every design works efficiently for every network. Keep in mind that I’m writing this here while thinking about this small office, and so many companies I’ve worked with that have offices of similar sizes.

Unfortunately, times are changing and this separation of router and firewall is no longer the best direction for small sites. After a few quick searches you can see that more and more threats today come from inside the network. New technology concepts such as BYOD, IoT, web proxies and private VPN’s are all technical contributors to this problem. Especially considering the human factor, administrators should no longer completely trust internal devices. It is too easy for a user to take home their work laptop home and come back into the trusted network where a new virus on that machine can spread un-checked. The typical IT organizations managing these smaller businesses no longer have reasons to allow this to happen:

  1. High-performance network devices are common and affordable; performance on the network cannot be a reason to not implement Next-Gen Firewall (NGFW) protection. Throughput on today’s hardware with NGFW features enabled can easily be greater than 1Gbps while still being very affordable, even for small businesses.
  2. Network availability is not a concern as any business-grade equipment from a reputable vendor should support HA capabilities. Insist on stacked switches for redundancy behind those firewalls? Great! Go for it. Just don’t let those switches be your internal layer 3 routers.
  3. Firewalling should be more than just blocking and allowing ports on the network. Here is the big differentiator between your common router and your NGFW firewall: the router with an ACL is only going to block ports/IP addresses. A firewall of course has this capability, but adds user identification, antivirus, intrusion detection, application control, DDoS protection, and more. If you’re saying to yourself that you’re fine with your Cisco 2900 router because you have ACL’s between your VLANs, you’re wrong. If you want to keep them that is your choice; maybe add a transparent firewall in there too, though.

Lets take network security to the next level. Don’t assume that yesterday’s network design is still the best fit for today’s world. And don’t assume that your inside devices are trusted! Take steps to protect your network at every level. That’s next-gen thinking.

Why I hate McAfee (the company) and why you should, too

Companies have a tough time fighting spam. I get it. Spam fuels the spread of viruses, phishing, identity theft, and general user confusion. I despise it as much as the next guy, but it has quickly become a part of day-to-day life with any email user or mail-enabled organization. Because of how rampant and aggressive spam email has become, as well as the ever-increasing danger of websites that spam may try to lead you to, companies that fight spam have taken up blacklisting: adding email domains and server IP addresses to one of several lists that are used be various spam filters to more easily detect spam emails. Getting on one of these blacklists can be entirely too easy, and oftentimes it is entirely too difficult to be removed once on one.

At this point you’re probably thinking “good, lets stop as many of those spammers as we can!” Well, the problem is that legitimate email-sending companies can get added to these lists. Before anyone knows what is going on, a legitimate and honest company is having problems sending (or even receiving) emails and business starts to grind down to a halt. At this point the company’s IT resources will begin sorting out the issue and eventually begging and pleading for their server to be removed from one or more blacklists that is crippling their email service. Some of the blacklist providers offer a simple web-based removal process that requires just a simple explanation… but then there is McAfee.

McAfee has a ‘special’ group within their anti-spam division, known as McAfee Messaging Security. This group, from what i have gathered, takes recommendations from affiliate organizations of domains that should be flagged as spammers and arbitrarily adds them to their blacklist without any sort of verification or validation of an actual offense. The only way to be removed from McAfee’s blacklist? Send an email to [email protected] or [email protected] and wait for them to tell you how they picked your domain randomly out of a hat and blacklisted it for no reason.

What’s the real problem, you ask? The real problem is that this Messaging Security group is ONLY AVAILABLE BY EMAIL! No phone call can reach them, no tech support case (even with Gold Support) will be escalated to them, EMAIL ONLY. So while your business is stagnant, crippled, and waiting for McAfee to get back to them to resolve the issue, your customers are fleeing, getting bounce-backs, and wondering why they aren’t receiving prompt replies. But wait, there’s more.

McAfee Messaging Security likes to keep things as vague as possible, that way you have trouble telling that they have no real reason for blacklisting you. Their first response to your email will be “uh, well, this website here has junk html files that need to be removed before we consider removal” (you may think I’m exaggerating, and I wish i was… this is how it actually happens). So, five email exchanges later (12 hours in between each one, mind you) and hopefully you’ll have the problem fixed, or at least have an idea of what you actually need to do to satisfy these ruthless email dictators. Hopefully the affected company won’t also be a subscriber of McAfee’s cloud-based spam filter, because if it is then the email replies from Messaging Security could even be caught in their own spam filter and the exchange could take even longer.

I’ll stop here with my rant. Hopefully you get the picture and take warning. McAfee produces sloppy, sub-par software and backs it with even worse service and support. McAfee is one company that I will never recommend to peers and customers for these reasons.

Data Backups: Can you rely on tape?

Almost every small business that I’ve had a chance to work with has used tapes as their primary means of backing up data (the others have had no backups or no data to backup). Tapes are used widely for several good reasons: they can be stored and archived for many years, they are fairly inexpensive, easy to rotate, and many backup software packages are designed around the use of them. But have you ever had to restore from tape? It’s a tedious process. First, you have to find the tape (or tapes) that has the data you want to restore from (hope you’re labeling them well!), then you have to catalog it, then you have to mount and restore it… something that should be so simple can actually be an excruciating process. After all of that (and the time it takes if you’ve been there) you had better hope that the backup was tested, otherwise it may not even restore properly. What if your business is completely down until the restore is done? That would make for a very stressful day.

In my opinion, tape is a legacy technology. Disk and flash storage is so affordable these days that companies can easily purchase storage capacity that can exceed the amount of space that they have with tapes. This can translate to faster and more reliable backups, longer retention periods, and more available space for future growth. Restores are usually completed faster, too. Interested in a hybrid model? Tapes can still be a good means of off-site archival. Or, you can look into cloud based backups options or even offsite disk-based backups. The options are out there, but you have to make the choice for the backup solution that is best for your business.

Did you know: Standardized Risk Assessments

Did you know that the National Institute of Standards and Technology (NIST) publishes Special Publication 800-30, Guide for Conducting Risk Assessments? This is one of several guides that GROUND Security incorporates within our assessment and analysis framework.

You can find the document on NIST’s website: http://csrc.nist.gov/publications/PubsSPs.html

Simple Security Tips

Some useful tips for home users to help stay protected on the internet:

http://yourhub.denverpost.com/parker/protect-your-identity-defending-your-internet/vGVQ6vjQnuGobG0KjFSgwI-ugc?hl