I am just now finishing up the first of a handful of example apps to help tease out the approach, tooling, and ClojureScript patches needed to make Clojure a viable option in this regard.

git-ttt

git-ttt is a git-backed, text-based, ticket tracker written in ClojureScript, making use of EDN as the storage format, supporting datalog-like query driven by core.logic, running on Node.js.  I pushed an early glimpse of it to github and published an intro screencast to allow those that are curious to see it in action.

Two interesting pieces of the project are its use of protocols and the approach to avoiding constant callbacks.

Protocols

“Program to an interface, never to an implementation.” and “Favor composition over inheritance.”  – Gang of Four

Protocols enable developers to open their systems for extension, while also allowing others to compose targeted functionality ala carte.  Shoreleave’s pubsub abstractions and git-ttt’s search abstraction are two concrete examples to see these benefits.  In Shoreleave, one can extend the full functionality of the pubsub to any object in ClojureScript¹ and extend the pubsub bus abstraction to any implementation.  Participation in this system is open to all, from all directions.

In TTT, the search protocol specifies a function to compare two attributes, while falling back on equality in the default case.  This allows plugins to extend search functionality and behavior.

Blocking Deref

One challenge when working with ClojureScript is staying true to Clojure style and programming with values, while also conforming to JavaScript’s callbacks.  One strategy in the browser is isolating those cases and placing them at the end of a threading macro.  Node rests its foundation on callbacks, so a strategy built around promises becomes useful.

Promises within JavaScript provide freedom from the constant callback concern, enable proper exception handling within the context of the exception, return the programmer to focusing on values, and allow for styling and composing ClojureScript like traditional Clojure code.  The pattern I’ve fallen back on is a combination of dynamic vars, promises, and a function called blocking-deref.

Scripting applications usually have some global constants (for example a file path string), which you might need to set from an external system call.  A more concrete example- TTT needs to know some of the user’s git settings, which are fetched with the git command via Node’s system exec call.  The result (a map of git config keys to their values) is bound to a dynamic var and the rest of code is written against that dynamic var.  To actually get that result though, we use an atom, a promise, and the threading macro.

First we write the system call to git in a standard callback style, except an atom will contain the eventual result.

(defn git-config-atom
  "Call the `git config` command to grab the value of a given key.
Return an atom that will hold the return/output string in a map {k output-value}.
Optionally pass in an atom (allowing you to build up many returns)"
  ([k]
   (git-config-atom k (atom {})))
  ([k ret]
   (do (sys-exec (str "git config --get " (name k))
                 (fn [_ out _]
                   (swap! ret assoc (keyword k) (cstr/trim out))))
   ret)))

Multiple requests are threaded, passing the atom along and building up the data.

;; This is the direct Result object you can use to access
;; a git config map
(def git-config-res
  (let [res (goog.result.SimpleResult.)]
    (->>
      (git-config-atom "user.email")
      (git-config-atom "user.name")
      ((fn [a] (blocking-deref a res #(< (count %) 2)))))
    res))

A promise awaits the values in the atom.  Blocking-deref just ticks away Node’s internal process until the promise is fulfilled.

(defn blocking-deref
  "Given an atom and a Result object, attempt to cycle the process tick on derefing until `pred-fn` isn't true
By default, `pred-fn` is nil?
Once the condition doesn't hold, the Result will be set to @a, and @a is returned"
  ([a r]
   (blocking-deref a r nil?))
  ([a r pred-fn]
   (if (pred-fn @a)
     (node/process.nextTick #(blocking-deref a r pred-fn))
     (do (.setValue r @a)
       @a))))

This overall approach is similar in notion to how transients work.  From a top-level, the value of the promises set the dynamic vars or are passed in before kicking off the main application loop.

(defn -main [& args]
  (let [git-root core/git-root-res
        git-conf core/git-config-res]
    (result/wait git-root
      #(binding [core/*repo-root* (.getValue git-root)]
         (result/wait git-conf
           (fn [] (main-control git-conf args)))))))

Scripting Clojure apps on Node.js is a real possibility and a complete pleasure by using the platform (Google Closure), the host (Node.js), and the language cooperatively.

1. By default Shoreleave extends the abstraction to functions and atoms, allowing construction of reactive dataflows

At the upcoming Conj, I’ll be speaking about Production ClojureScript – and this time I’m bringing the data

The presentation discusses some features of Shoreleave, the suite of utilities Tutorspree built up for building ClojureScript apps.  The slides also provide some suggestions on how to build out or structure your ClojureScript applications.

Coming soon are two example apps (one of which was demo’d during the talk) – a SOLR-connected CLJS client a TodoMVC example.  Both will come bundled with a teaching/exploration tool (which was also demo’d last night).

Once the demo apps are done, I’ll be doing a quick screencast on getting the most out of Shoreleave.

Lastly, thanks to everyone who has help shaped Shoreleave and provided me with feedback.  It’s all greatly appreciated.

The goal is simple: Continually be learning something new.  Experiment and explore the new material or technology.  Integrate the knowledge or technology and use the integration as the platform to launch into a new learning area.  Following these three steps will allow you to quickly evaluate technology and effortlessly adopt it.

Grok It

Continual learning is essential for producing an ideal solution to a problem.  Getting in the habit of constantly reading, discovering, evaluating, and discussing new technologies grows your toolbox of approaches and maps out solution spaces for classes of problems. Deciding what to learn and how much to learn is tricky.  Follow your own curiosity or try Just Enough Learning.²
Identify areas of your problem that are high risk (if you get them wrong, your entire solution is guaranteed to fail) and areas of deep impact (where an ideal solution greatly reduces the overall problem).  Spend no more than one hour quickly searching Google, Google Scholar, and tech news resources.  Identify the few ideas, concepts, or tools to Rock.

Rock It

Pick the technologies up and USE THEM.  Go beyond recreating an example.  Use the technology to solve a subset or possible subset of your problem.  Your goal should be understanding the benefits, tradeoffs, and limitations to the technology.  This is raw validation by means of a prototype.

Drop It

Integrate.  Right now, do it.  Integrate the knowledge you gained in better focusing your own approach or integrate the technology.  The more often you integrate smaller pieces that simplify your problem, the quicker you’ll arrive at an ideal solution.  The natural byproduct of integration is a new perspective to tackle another area in the problem space, launching you back into a phase of learning and exploration.

1. This is greatly stressed in Polya’s How to Solve It
2. This term is glommed from the book, Just Enough Software Architecture

Engineering Cultures

There are many examples of the deep impact positive cultures have on their populations, no matter how large or small the population is.  For example, take Do-It-Yourself (DIY)- the culture and camaraderie within it have existed for decades.  It’s become a way-of-life for some individuals.  The culture is so powerful, it has even been adapted into a personality trait ala I’m a real do-it-yourself’er.

Similar effects can be found in technical organizations and even within software projects.  Ruby has had a profound effect on the approach and adoption of software testing.  TDD is the norm for Ruby developers (especially those using Rails), to a degree that wasn’t previously seen in other languages or projects.  Likewise, I’d argue that Clojure is causing a “Thinking” culture to emerge- where careful, deep, purposeful thought and dissection of the problem space are the first steps of all ideal solutions.

This concept of “purposeful culture” can be leveraged to boost your own engineering team.

A tale of engineering at Etsy

I credit my experience at Etsy as the first great example of how powerful a team/company culture can be and what results given one.  At the earlier days of Etsy, everyone was more or less equal.  The entire company (at the time I left, 65 employees), was roughly a horizontal organization.  Every idea and criticism was valuable and received the same amount of attention- as long as you had the data to back it up.
We decided as a team (and to some degree, as a family) what was important to us, what we thought we should work towards, and how we should go about doing that.  Employees were valued above everything else.  Being in a room where everyone was a first class employee not only allowed us to get to the winning ideas faster, but made everyone feel important.
Data and democracy ruled.

It didn’t stop there either.  Etsy has continued this tradition of optimizing for the employee and driving all their decisions by real data.  Chad Dickerson sums it up nicely in a talk he delivered at railsconf.

Culture’s effects on process

Effective process comes from effective cultures, not the other way around.
If you want the best possible process for your team, you must first create a culture where the natural outcome is that process.  Here are some things that worked for Etsy and have continue to work for me as I’ve built out engineering teams:

• Strive to be objective, always.
• Give everyone access to all points of data
• Encourage innovation and create clear paths for bringing the fruits of labor before the group
• Drive ownership across everyone
• Continual and constant learning
• Mistakes are ok, Failures are ok, Adaptation is essential
• Ask “why” more than “what”

1. Much work has gone into studying the network and behavioral effects of populations given different societal inputs.  See Networks, Crowds, and Markets by Easley and Kleinberg
2. Here I’m using powerful in the terms of “work effort over time,” based on an assumption that social interaction amongst people is solving a specific problem.  In the same regards, I’m using best to refer the lack of software/hardware/”systems” engineering in the elegant solution.

• Every piece of software has an architecture, whether intentional or not
• Every architecture is created by at least one architect
• Architecture appears in the small (software design patterns) and in the large (architectural styles)
• Architecture is independent of technology choice.  Abstract styles allow for composability
• Architectures are objective, allowing us to say one architecture is more elegant, efficient, or better than another

This installment will cover how to recognize the styles being applied in an architecture through a process of analyzing for completeness, consistency, compatibility, and correctness.  The post will finish with how to evaluate an architecture after it’s gone through appropriate analysis.  The main goal here is to use purposeful, objective decision making and problem solving skills, to move forward in building the best possible system and enable us to have a higher chance of software project success.¹

Analyzing an architecture

Most often architectures need manual analysis, however, there are automatic and semi-automatic analysis techniques and tools.²  I’m primarily going to be discussing how to use general inspection and review based analysis and specifically Architectural Trade-Off Analysis Method (ATAM).  Both of these methods are manual, involving discussion,³ and usually include iteration.  As always, this is just a foundation for these two methods, please feel free to adjust them for your own purposes or seek out more resources.

Terms and goals

Here is simple language used discuss and analyze architectures in an objective way:

• Completeness – The system captures all functional and nonfunctional requirements.  The architecture itself fully captures all views and components of the system, as well as dynamic behaviors, interfaces, and contracts.
• Consistency – The absence of contradictions within the system.  These can appear as naming or interface inconsistencies with components or connectors, behavioral inconsistencies between components, and interaction or protocol inconsistencies.
• Compatibility – The system correctly applies and adheres to the constraints of the styles and reference architectures.  That is, you’re not trying to compose styles in an inefficient or incorrect way.
• Correctness – A relative property that refers to if an architecture captures all design decisions and if a system’s implementation fully captures the architecture, correctly implementing all design decisions

When analyzing an architecture using this language, highlight:

• The data exchange within the system or subsystem – most systems (especially web-based systems) are data-intensive or data-centric.  The structure of the data, the flow of data, and its general properties and characteristics should all be analyzed
• The structural characteristics of the architecture, ie: how everything connects or if components are missing necessary connections
• The behaviors of components
• The interaction between components
• The nonfunctional characteristics of all components and connectors – often the critical dimension of all software systems

Given this language and framework, architectures can be objectively inspected, discussed, compared, and analyzed.

Inspection and review

An architecture inspection and review can be thought of like a code review, but for the system’s architecture and design decisions.  Just like in a code review, quality and knowledge transfer are the two major takeaways from the process.  That is, inspection and review will find defects and shortcomings, better address design decisions, and allow all team members to fully understand the entire system’s architecture.  The scope, goal, and concern of the review is determined by the team and project, but the process is the same no matter what.

Individuals first conduct their own review, creating whatever artifacts (models, notes, written assessments, reference architectures) they feel they need.  A group then conducts a review together guided and supported by the individuals’ reviews.  This can be done in any fashion, but there are specific formats, such as the ATAM.

Architectural Trade-off Analysis Method

The details of the method can be found at the Software Engineering Institute’s site but a graph will be sufficient:

ATAM's flow

ATAM centers on a scenario based approach, which include:

• Use-case scenarios – how the system is envisioned to be used
• Growth scenarios – planned and envisioned modifications to the system and architecture
• Exploratory scenarios – the limits of the architecture’s adaptability when given major changes to the functionality, operational profiles, and underlying platform

ATAM then uses technical constraints, external and integrated systems, and architectural approaches, to find the best possible match between the scenarios (which determine quality attributes of the system while expressing the requirements) and the best design decision.  ATAM is useful in that it inherently includes capacity planning, which is critical for most web systems that get deployed.

Evaluating a candidate architecture

It’s sometimes helpful to have a tangible working artifact when designing, analyzing and evaluating an architecture.  Creating a prototype or architectural skeleton are two ideal techniques, but are not substitutes for the methods described above.⁴  A prototype is useful in understanding how an architecture or reference architecture “holds together” or “stands up.”  For some architects, it’s an easier visualization of multiple components and connectors without getting lost in the functional and non-functional requirements.  As a consequence, prototypes prematurely force you into technology choice before a real architecture can be distinguished.  Prototypes come in two major forms:

• Horizontal prototype – which models wide aspects of a single layer, i.e. many features with little details. The most common example for Horizontal prototype is a  user interface prototype which is used to test the overall interaction with the system.
• Vertical prototype -Implementing some sub-system or a limited set of features across all layers /modules.

An architectural skeleton is the first iteration of a candidate architecture, trying to loosely tie together the various technology choices that make up your components and connectors.  I strongly suggest all architects do this, as it gets you to a useable product early on and allows for experimentation and exploration within a live version of the architecture.

1. A successful project is most often defined as being on time, under budget, meeting all functional and nonfunctional requirements, and being constructed in a way that enables ease of maintenance and desirable evolution.
2. Automated analysis usually comes from using formal or semi-formal architectural methods, like xADL or one of SEI’s methods
3. Any sort of collaborative communication will work, I prefer a wiki because it documents the history of the analysis nicely
4. These terms were lifted from Arnon Rotem-Gal-Oz’s blog

A system’s architecture is an expression of its characteristics.  It is the combination of all design decisions, tradeoffs, and constraints.²  The validity of an architecture is determined by objectively evaluating how its design decisions and behaviors map to the system’s functional and nonfunctional requirements.³  Let’s walk through some core concepts about software architecture and then enumerate various evaluation strategies.

Every software system has an architecture

Every piece of software has an architecture, whether intentional or not, and was created by at least one architect.  This idea can be seen in other forms of architecture, for example, every building has an architecture.  Not all architectures are elegant or complete, but nonetheless exist.

Architecture appears in the small and in the large

Software architectures are often split up into two categories.  Architectural Styles are named sets of design decisions that result in superior system properties, yielding something that is more “elegant, effective, efficient, dependable, evolvable, and scalable.”⁴  These decisions are made at the system level and are abstract.  Some examples are client-server, REST, pipe-and-filter, peer-to-peer, and DDS.⁵

Architectural Patterns, often called Design Patterns or Software Design Patterns, are named sets of design decisions that result in superior properties for a component or within a component.⁶  These patterns are usually concrete or partially concrete implementations.⁷

Architecture is independent of technology

Architectural styles are abstract and aren’t tied to any particular piece of software, technology, or language.  Doing so adds unnecessary constraints on the design,⁸ similar to the result of premature optimization.⁹  These additional constraints will at best reduce your system’s efficiency and at worst prevent your architecture from being valid.  Additionally, architectural styles need to be abstract to enable composability.

Architectures are objective

Architecture is a continual effort throughout the entire life cycle of software.  A system’s architecture evolves as your team elicits new requirements and gains information in your problem domain.¹⁰  Characteristics of an architecture are objectively evaluated, enabling us to say one architecture is more elegant or efficient than another.

Your goal in design and creation is to purposefully apply design decisions, using established styles and patterns, to achieve the best architecture possible. A necessary step in effective problem solving is critically analyzing all possible solutions, and objectively determining the best one.  Evaluating architectures is a software engineering equivalent.

Next installment and further reading

The next part of this blog post will use examples to discuss:

• Recognizing styles
• Analyzing an architecture’s completeness, consistency, compatibility, and correctness
• Applying architectural components to achieve nonfunctional properties
• Evaluating conceptual architectures

A third and final post will use those skills to design a “real time” web system, like Gmail or Twitter, starting from requirements and delivering a valid architecture.

Interested readers will find more information in Software Architecture: Foundations, Theory, and Practice.  I strongly recommend the book to all software engineers as a great reference.¹¹ Arnon Rotem-Gal-Oz’s Blog is a solid source of higher-level software architecture discussions.  High Scalability and Web Performance Matters are two wonderful sites about internet architectures.

Lastly, reading and studying well composed architectural styles will make you a better architect.  Applying the same lessons when faced with similar requirements or constraints will produce a better systems.  The Architecture of Open Source Applications is great, recent source of architectural walk-throughs of some of the most popular, successful, and widely adopted open source projects.

1. In a later post I plan on appropriately defining the field of software engineering, as I most often make use of it.  I feel like some people needlessly narrow the scope of “software engineering” – I try to somewhat establish a definition in this comment.  The Parnas paper mentioned in a previous post details it better than I ever could.
2. Understanding an architecture is the pursuit of exploring and understanding those decisions.  Reviewing an architecture is similar to reading and reflecting upon someone’s biography or recognizing the quality of construction in a new building or automobile.
3. Nonfunctional requirements are often called the -ities. Security, scalability, dependability, adaptability, complexity, efficiency, and robustness are some of the more common nonfunctional requirements.  Being explicit about these requirements allows for a more complete architecture and better visibility with regards to testing/verification
4. Software Architecture: Foundations, Theory, and Practice. Taylor. Dashofy.
5. I used these styles because they all map to widely used successful system.  Respectively, the Internet, web services, Unix – (including Mac OS X), bit torrent, radar systems and emerging “real time” web.  DDS is data-centric specialized form of another style, publish-subscribe
6. Design Patterns are the expression of desired functionality, within the constraints of given language or paradigm (and are therefore language or paradigm dependent).  For example, many of the classical design patterns are obsolete or do not apply in functional languages, largely because they were written for Object Oriented languages.
7. Some pieces of architecture fall between both categories, like Model-View-Controller, which is an architectural style that is implemented in frameworks for various languages.
8. The constraints are the result of passively accepting the tradeoffs inherent to certain technologies
9. In software, premature optimization is often a root cause to inhibiting system flexibility, the cause of integration problems, and a telltale sign of an inexperience software engineer
10. Architectures help establish knowledge by applying information from a problem domain into a solution’s architecture
11. The book features a matrix that identifies how each style can be composed, and the system properties that result from such a composition.  This matrix is worth the book price alone.

Must-read list of software engineering books

The Mythical Man-Month – Frederick Brooks
Software engineering is an objective practice, entangled with people and process.  This book offers great advice on how to approach managing software projects and the practice of software engineering.

How to Solve It: A New Aspect of Mathematical Method – George Polya
“Solving problems is a practical art.”  This book does a great job of detailing the process and tools to decompose problems and build up ideal solutions.  It’s targeted at mathematics, but the subject matter is very much universal.

Information Technology Project Management: Providing Measurable Organizational Value – Jack T. Marchewka
This entire book is practical in its material, none of it is theoretical.  It covers metrics, risk analysis, estimation techniques, and managing cycles.  I always reach for this book when spinning up new projects.

Software Architecture: Foundations, Theory, and Practice РTaylor, Medvidović, Dashofy
I have recommended this book in the footnotes of previous posts.  This book walks through architecture styles, composition, appropriate ways to approach designing, validation techniques, and more.  The composition matrix in the beginning chapters of the book is worth the purchase alone.  To boot, Nenad Medvidoviƒá is a professor with Barry Boehm.¹

The Architecture of Open Source Applications – Edited by Amy Brown and Greg Wilson
This book is FREE to read online.  It carefully walks through the architectures of some of the most used and successful open source projects, the tradeoffs made, and the dirty hacks that hold everything together.  Working through this book will allow any engineer to apply the lessons learned when faced with similar situations or constraints.

Interaction Design – Jenny Preece, Yvonne Rogers, Helen Sharp
A great book about user-centered interaction and experience design.  This book provides the tools to design, prototype, validate, and evolve interactions and interfaces for your systems.

Hacker’s Delight – Henry S. Warren
Hacker’s Delight is a collection of efficient, space-saving, performance hacks for common programming tasks and system operations.  I haven’t put a lot of the material of this book to use, but it’s useful to have and interesting to paw through.  This book has little to do with the practice of software engineering, but still should sit proudly on your shelf.

Seven languages in seven weeks – Bruce Tate
Learning and applying multiple paradigms allows you to find best tool for the job.  Opening your engineering-vision to what is possible will enable you to engineer more elegant systems.

Must-watch list of software engineering videos

Are we there yet? – Rich Hickey
Has Object-Oriented programming delivered on the software engineering principles it promised?  Are we engineering better systems because of OOP, or have we missed the mark?

Hammock driven development – Rich Hickey
There is value often overlooked in working through a problem domain, properly decomposing it, and validating your proposed solutions early.  This talk is about how to solve problems better and the steps to do it. ²

How to design a good API and why it matters – Joshua Bloch
This Google Tech Talk video talks about designing, validating, and building out APIs, but the lessons here are universal to all interfaces and public-facing solutions.

Honorable mentions

Erich Gamma – I didn’t select any of his works, but some of his guiding principles are the most important in software engineering, namely: “Program to an interface, not an implementation” and “Favor composition over inheritance.”

Paul Graham – Aside from authoring great books about Common Lisp, Paul Graham has elegantly described and carefully organized works and efforts around Hacker Culture.  If you are a professional software engineer, you have to own Hackers and Painters.

Missing from the lists

Validation and verification is an essential part of the software development life cycle, but I didn’t list any books covering the topic.  The simple fact is that most software testing books are terrible.  If I had to recommend one, I’d probably offer Software Testing by Ron Patton.

I’m also not wild about any book I’ve read on requirements elicitation and specification.  The only one I could really recommend is Managing Software Requirements: A Use Case Approach by Dean Leffingwell and Don Widrig.

People’s Choice

Many people sing Russ Olsen’s praises, especially his book Design Patterns in Ruby.³ If you work in any OO language, this book will open your mind in how to best apply design patterns.

1. Boehm was the author of some of the most influential papers in software engineering
2. There is a useful cheat sheet graphic.  I will temporarily make it my desktop image when I begin working on a new problem
3. All of Russ Olsen’s books on Amazon have 4 or 5 start ratings

In a follow-up post I will present a list of books and videos that I have found useful in architecting and building software, or that sharpened my approach to software engineering.

Must-read list of software engineering

These are grouped by author and in no particular order.  Take your time and carefully digest each paper.  You can optionally download all the papers as a .zip file.

Software engineering programmes are not computer science programmes²
If you’re going to call yourself a software engineer, you should mean it.  This paper concisely lays out the difference between the two and the importance in shaping education around the former.³

Designing software for ease of extension and contraction
This paper discusses the importance of loosely coupled, highly cohesive modules, that only expose specific functionality.  The root goals of object oriented programming were based on this paper.

Software aging
Software is an evolving system.  It ages just like bridges, buildings, and cars.  This paper was the first to discuss how to approach software evolution.

Software project management practices: Failure versus success⁴
This is a case study on why certain projects fail while others succeed.  There are only a few common reasons why projects succeed, but they are essential for success.

Software metrics: good, bad and missing (I can’t find a freely available copy of this article)
Software and the process to create it are objective matters.  We can measure them and say whether they’re good or bad.  Metrics give us a baseline to make these statements, and this paper presents a holistic approach.

No silver bullet: Essence and accidents of software engineering⁵
There is no silver bullet for software engineering, but there are core concepts that help us engineer better systems.  The details and examples in this paper are dated, but the root concepts still hold.

Software engineering economics⁶
Estimation is hard and is exacerbated by our lack of understanding of a project’s remaining work.  What follows is a survey of various estimation techniques and their accuracy.  Boehm then introduces COCOMO.⁷

Software risk management: principles and practices
This paper discusses the importance of risk management in a general software engineering process, as well as how to implement for your team or project.

Verifying and validating software requirements and design specifications
Starting the verification and validation process at the point of doing requirements will remove more problems within the system faster, cheaper and earlier than conventional practice.

1. I’m using better to mean increasing the depth of one’s skillset, focusing more on objective driven engineering, and using skills and techniques known to produce software that is more secure, scalable, manageable, maintainable, etc.
2. This paper and the three that follow are by Parnas, considered the grandfather of software engineering education, object orientation, data encapsulation, and general large system design
3. Please note that I received my B.S. in Software Engineer, so assume some bias
4. This paper and the following one are by Capers Jones.  He has contributed well conducted case studies and publications focused on metrics, risk management, requirements engineering, and objective estimating.
5. This paper is by Brooks, who wrote mostly about theory vs practice and software project management
6. This paper and those that follow are by Boehm, best known for his contributions to estimation, risk and requirements management, the creation of the spiral or iterative development life cycle (now widely used in Scrum and other agile methods), and measuring and estimating the cost of software
7. When estimating projects, I often use COCOMO (or some variation of it) as my top-down estimation technique.  I then independently use a bottom up estimation technique, and go through a process to reconcile the two.