Uncategorized

Mac OS X Server Time Machine Volume Filling Too Fast

I’ve been noticing on my Mac OS X Mavericks 10.8 Server I have running in a data center has been filling up its Time Machine volume way too quickly.  The backups are continually huge and only about a week fits on the second hard drive inside of the Mac mini.  Every time the machine backed up it was taking up so much room that previous backups had to be deleted.

I searched everywhere and couldn’t find an issue.  I looked in console logs for Time Machine and couldn’t find anything definitive as well but it looked like it was something in /Library/Server.  I realized that it must be a file or set of files that are always being written to so they’re always backed up every hour or so.

I had tried some command-line tools like “du” but the usage was so many levels deep it was hard to pinpoint.  Finally I grew a brain and installed DaisyDisk on the server to see what I could find.  Within minutes I found the pinch point.  See all of the repeated pattern in the third layer from the center?

2014-08-21_07-53-41

Turns out the file /Library/Server/Xcode/Logs/xcs_proxy.log was huge and never rolled over by the logging utility.  I had been using Xcode on this server to run Bots but had since turned it off.  I still need to figure out WHY this file is being written to without the Bots service running.  Because the file is always being written to, the entire thing is backed up every single hour.

Solution:

sudo rm /Library/Server/Xcode/Logs/*

If I find the solution to the xcs_proxy issue, I’ll follow up.

Standard
Uncategorized

Asynchronous unit testing Core Data with Xcode 6

The WordPress for iOS project had a number of unit tests using Core Data and a custom asynchronous test helper.  The helper used a semaphore in a global scope and a bit of method swizzling to give a wait/notify mechanism.  The problem with this solution was the global semaphore and poorly written tests causing a conflict.  Tests would call the ending wait and previous tests running Core Data would fire off notifies causing a mismatch between the original test and the recipient of the message to pass by the current semaphore.

The solution was to eliminate the global scope and have a semaphore object/instance that could be passed around by the single test.  The more I started designing the solution I realized we should just start using the new asynchronous testing capabilities of Xcode 6.  Xcode 6 / iOS 8 / Yosemite brings us a wealth of new tools to play with including Swift, the newest language developed by Apple.

XCTestExpectation

The newest member to the XCTest framework is XCTestExpectation.  This class encapsulates a single semaphore (of sorts) allowing your test to wait for 1..n things before the test is considered over.  This helps with asynchronous things like AFNetworking, Core Data, and Grand Central Dispatch.  If you’re using a Core Data stack with multiple NSManagedObjectContext instances (background & main for example) then you’re going to almost certainly want to know how to test asychronously.

Creating a XCTestExpectation is quite simple:

- (void)testObjectPermanence {
    XCTestExpectation *saveExpectation = [self expectationWithDescription:@"Context save expectation"];
    // ..
}

The method expectationWithDescription: is part of XCTestCase and it has a companion method waitForExpectationsWithTimeout:handler: which you call when you’re reading to block the test execution until [saveExpectation fulfill] is called.  Fulfilling an expectation means the expectation has been met and the flag should be flipped for the test that is waiting.  If your test execution never reaches a fulfillment, then your test will fail when your timeout threshold is met.

Some important things to note about XCTest asynchronous testing:

  • A XCTestExpectation instance can only be fulfilled a single time.  Use multiple instances if you have more than one expectation.
  • Only a single waitForExpectationsWithTimeout:handler: can be active at any one time.
  • Unit tests, by default, run one at a time and not in parallel.  If you modify the test suite running to parallelize, I can see things breaking fairly quickly.
  • There are convenience methods to create expectations on a KVO property or NSNotification

Core Data Secret Sauce

There are a bunch of waits you could fire off a fulfillment on an XCTestExpectation when a Core Data context saves.  In the case of WordPress-iOS we use a central class, ContextManager, to help manage all of the Core Data levers and knobs.  There is a companion class called CoreDataTestHelper that does a bunch of useful things like overriding our persistent store to be in memory rather than on disk and also swizzle a method on ContextManager to fire fulfillments if an expectation is recorded.  Here’s an example of a test using this expectation helper:

- (void)testObjectPermanence {
    XCTestExpectation *saveExpectation = [self expectationWithDescription:@"Context save expectation"];
    [CoreDataTestHelper sharedHelper].testExpectation = saveExpectation;

    NSManagedObjectContext *derivedContext = [[ContextManager sharedInstance] newDerivedContext];
    Blog *blog = [self createTestBlogWithContext:derivedContext];
    [[ContextManager sharedInstance] saveDerivedContext:derivedContext];

    // Wait on the merge to be completed
    [self waitForExpectationsWithTimeout:2.0 handler:nil];

    XCTAssertFalse(blog.objectID.isTemporaryID, @"Object ID should be permanent");
}

You can see we store the single expectation for the next NSManagedObjectContext save on the CoreDataTestHelper sharedHelper instance. Later on when the context is saved, the following swizzled code executes and fulfills the expectation:

+ (void)load {
    Method originalSaveContext = class_getInstanceMethod([ContextManager class], @selector(saveContext:));
    Method testSaveContext = class_getInstanceMethod([ContextManager class], @selector(testSaveContext:));
    method_exchangeImplementations(originalSaveContext, testSaveContext);
}

- (void)testSaveContext:(NSManagedObjectContext *)context {
	[self saveContext:context withCompletionBlock:^() {
        if ([CoreDataTestHelper sharedHelper].testExpectation) {
            [[CoreDataTestHelper sharedHelper].testExpectation fulfill];
            [CoreDataTestHelper sharedHelper].testExpectation = nil;
        } else {
            NSLog(@"No test expectation present for context save");
        }
    }];
}

Next Steps

Our asynchronous tests are definitely not finished and the Core Data testing stack will most likely continue to change.  Our service layer doesn’t enforce a single save paradigm – we tell contributors that they should probably save often and at least one time before the service method completes.  The problem with the design we’re using above is that there is no good way to let a number of saves happen and to wait until the very last save.  In order do that, we’d have to offer up a special saveContext method on ContextManager that could get an instance of an XCTestExpectation.  That method would exist only as a class extension in our CoreDataTestHelper.  It’s messy but then again so is Core Data at times.  :)

Pull Request

The changes I made to the WordPress for iOS project are up for review in this GitHub Pull Request.

Standard
Uncategorized

Things With Wheels Are Meant to Move, No?

Table

I met some fellow coworkers at Hudson Business Lounge downtown Milwaukee today on a $25 day pass.  I’ve been a member in the past but haven’t been there in over a year.  I sat down and realized that the four of us wouldn’t fit at the table because of that awkward power outlet under the one chair.  I decided to move the table so that it was centered between the chairs.

This is when it all went wrong.

As I started to push on the end on the right pictured above, I realized quickly that physics (as I knew it) wasn’t working right.  Instead of the table top moving parallel to the floor, it started to nose dive forward.  Suddenly I felt a sharp pain in my shin and saw my Kleen Kanteen coffee cup rocket off the table and onto the floor.  The two MacBook Pros looked at me in horror as they gripped on for dear life with the 34 cent rubber pads under them.

Thankfully the MacBook Pros stayed put and only one got kissed with a splash of coffee + soy milk.

That’s about when I realized the table top wasn’t actually affixed TO ITS LEGS.

Table modified

Coincidentally this meshes well wish my previous post about not assuming things were done correctly in the past.  Just because something looks like it works right doesn’t mean it actually does.

The best part is the guy working front desk told me to not feel bad, it’s happened before.

 

Standard
Uncategorized

I was apparently feeling philosophical the other day and posted two tweets about dealing with systems architecture.  They generally apply to life as well, so I’m posting them here on my blog so I don’t forget them.

Just because something was done a certain way, doesn’t mean it was done right and even if it was done right at the time it may be wrong now.

Never fear asking questions and challenging even yourself.

Wax on, wax off.

Systems Philosophy

Quote
1 year work anniversary
Uncategorized

One Year at Automattic

Today marks my one year anniversary with Automattic.  I work with some awesome people and it’s been a rewarding first year.  I’ve had challenges to face (my attention for one), fun speaking at conferences, shipped a number of improvements and new features of WordPress for iOS, and learned a ton of new things.  I’m excited to experience my next year here!

Standard
Uncategorized

Authentication improvements for testing your apps

Aaron Douglas:

Developers interested in contributing to the WordPress mobile apps can now connect to WordPress.com with their own account. More details to follow on how to set up the individual projects.

Originally posted on Developer Resources:

We’ve just made it easier for developers to authenticate and test API calls with their own applications.

As the client owner, you can now authenticate with the passwordgrant_type, allowing you to skip the authorization step of authenticating, and logging in with your WordPress.com username and password. You can also gain the globalscope so that you no longer need to request authentication for each blog you wish to test your code with.

This is especially useful to contributors of the WordPress Android and iOS apps, which previously required special whitelisting on our part.

Here’s an example of how you can get started with using both these features:

Note that if you are using 2-step authentication (highly recommended) you will need to create an application password to be able to use the passwordgrant_type.

1
$curl = curl_init( "https://public-api.wordpress.com/oauth2/token" );
curl_setopt( $curl, CURLOPT_POST, true );
curl_setopt( $curl, CURLOPT_POSTFIELDS, array(

View original 88 more words

Standard