Conscious Granularity

As our skills advance in creating software, we work hard to break problems down into smaller pieces. This helps manage complex problems by tackling them in digestible chunks. Should we be careful wielding this approach?

Interfaces

As developers we are constantly building interfaces. These interfaces could be API’s, services, package interfaces, object interfaces, JSON structures, or function interfaces. The boundary is any unit we choose to define.

When designing these interfaces, carefully consider the granularity. I’m guilty of not considering this, and I suspect others may be too.

The Example

In our real world example we are building an interface to publish databases. Databases are made up of tables, tables are made up of files, and these files all live in a cloud object store like S3. We apply our technique of breaking a problem down, and we arrive at the following steps:

1. For Each Table
   1. If Destination Table exists:
      1. Delete Destination Table files
   2. Copy Source Table files to Destination Table location
2. Perform Schema Discovery/Creation
3. Register The Database

Excellent, we have an outline of how to perform the job, and an idea of operations needed. Now we want to encapsulate this behind an interface, let’s design it.

Fine Grained Approach

As a developer, the first thing that jumps out to me are the delete/copy operations. Those have reusable written all over them. Here is our resulting interface:

def s3_copy(source, destination)
def s3_delete(key)
def s3_exists(key)

def discover_schema(table_key)
def create_database(name)
def add_database_table(name, schema, table_key)

Coarse Grained Approach

Alternatively, let’s go the coarse grained approach, and we end up with a single operation.

def publish_database(tables)

Which Is Better?

The answer to which is better is always the same: it depends. The important thing is to recognize the different options and consider the benefits and drawbacks of each.

In our case, while the fine grained approach is very composable, let’s consider the significant benefits of the coarse grained approach.

• Intent – I’m a big believer in capturing intent in code. The operation publish_database captures the intent, a variety of calls to 6 functions that might be spread out does not. If the steps of publishing a database change, there is an obvious place to do that.
• Implicit Responsibility – Our intent was to publish a database, but with the fine grained approach we have also assumed the responsibility of generic S3 operations. This include testing, support, documentation, and so on.
• Testing – There are fewer paths of execution to test with the coarse grained approach. The Fine grained approach is more open to different usages.
• Permissioning – Creating S3 operations was really enticing, because we frequently do those operations. There are a myriad of S3 permissioning problems that may crop up. With the coarse grained approach, however, we can tightly control the from/to locations and permissions.

Always be aware of the trade offs you are making.