Thanks to Mark Coffman: http://markcoffman.com/2008/05/27/linq-results-for-paging-in-a-datagrid.aspx

When using binding a gridview to a LINQ result, make sure you return it to a List<T> (I’m assuming any IEnumerable<T> would work, but I really like the List<T>) to prevent yourself from getting

The datasource does not support server-side data paging.

Note that this is for easy paging.  If you want to do true data-paging with LINQ obviously you need to utilize the Skip() and Take() methods to get back just the data you want to show.

But I’m lazy… :)

My biggest take-aways were all conceptual...All the problems we have with things "breaking" and being generally not (automatically) repeatable come from us not takng the time to mitigate certain common risks:

Risk Aversion

• Automated Testing is acknowledgement that coding bugs can and will occur.
  
  
• Versioning is acknowledgement that shared files can and will be overwritten.
  
  
• Agile is acknowledgment that Requirements Gathering can and will fail.
  
  
• Continuous Integration is acknowledgment that deployments can and will break.
  
  
• TDD is (in a way) acknowledgement that refactoring will constantly occur.
  
  
• ...

If you don't take these risks into account in your process, the problems will all end up getting solved individually for each project (ugh) or worse -- swept under the rug until it's too late. The more common risks we can acknowledge and plan solutions for, the easier our jobs will be.

Understand what you do so you can do it better

Presenter – Tim Rayburn (Sogeti)

• What is SOA? (Service Oriented _________)
• Approach
• Architecture
• Application
• Analysis
• Agreement
• Going to an SOA should NOT be changing how you do what you do
• SOA is philosophy, not product
• Aligning business with IT is the goal
• It is EVOLUTION, not Revolution
• Mapping Your Business
• Business should have an idea as to what their process is, preferably a process map
• Draw out what you want done including the weird scenarios
• Define what your organization does, and what your organization does only to get a certain job done (i.e. email is not what your business does, it’s something your business does to get your job done)
• SOA leads to Agility, rapid response to change
• Where to Start?  Choose wisely!
• Choose the right people to work with (not perfectionist, good enough is good enough)
• Types of Services
• Composite Services
• Packaged Application Services
• Custom Application Services
• Infrastructure Services
• Not a singular task, but a sequence of tasks that get something done
• Users don’t care about what happens on the back end, as long as the front end stays the same.
• Given the right input, we can make a service work, regardless of back end technology
• Infrastructure Services are NOT the goal, they are the tools to get the job done
• How to Think about Services
• Services are a black box, you get to care about inputs and outputs, but not about how it does the work
• Services are not always answered by computational algorithms
• Services aren’t always computers/machines, services can include human powered tasks, but must apply the same rules of input/output only
• Services can be departments where work is queued for a human
• Obviously these should be asynchronous

 

Presenter – Tim Peek (TenStep)

Fear with estimates is that they will be inaccurate, but that’s just what an estimate is – a guess.  You’ll only know if your estimate was right when you’re done.

A lot of the information to build estimates comes from historical data.  Keep estimated vs. actual to determine estimation success

Estimates also give a yardstick as to how well you’re doing.

ROI is all about estimates, comparing the estimate to the value of the end product

Estimate is a judgment of size, number, etc. that needs assessment

Estimating is concerned with the prediction of uncertainties.  It is more dignified than fortune telling, though not always more accurate.

• Project Estimating
  • Effort (NOT duration)
    • You can’t determine duration until you figure out the effort needed
  • Cost
  • Schedule
• Why is it important?
  • Supports decision-making
  • Provides an ability to deal with the unknown
  • Determines what actions we see as safe and risky
    • Hiring
    • Spending
    • Lifespan of equipment
• Preparing for the Estimate
  • Determine estimating goals and tradeoffs
    • Quality
    • Cost
    • Time
  • Document the prior estimating assumptions and risks
  • Distinguish between effort and duration
    • I can have this activity done in 2 weeks.
    • This activity will take 40 hours
• Levels of Accuracy
  • Determine if there IS a level of accuracy requested or required (+/- 50%, SWAG, WAG)
• Estimate the Effort

Estimate	Accuracy	Purpose
Order of magnitude (Conceptual)	-25%-+75%	Evaluation of projects or alternatives
Preliminary (Budget)	-15%–+25%	Establish initial budget, reserve funds for project
Definitive	–5%-+10%	Establish actual project budget, after Project Charter

• Preparing for the Estimate
  • Avoid estimating what you do not understand or know
  • Get a clear picture of the work you need to estimate
  • Determine if full life-cycle costs are required
  • Determine the resources covered in the estimate
    • Immediate project team
    • Clients
    • Vendors
    • Cross functional resources
• Estimating Techniques
  • Previous History
  • Analogy
  • Ratio
  • Expert Opinion
  • Delphi – Group of Experts via Questionnaire
  • Parametric Modeling
  • Work Breakdown Structure
  • PERT ((optimistic + 4x median + pessimistic)/6)
• Estimating Errors
  • You underestimate the size of the deliverables
  • You don’t include all of the activities required
    • Documentation
    • Project Management
    • Training
      • You commit to an estimate that is too low
  • You estimate based on building good quality in the first time
  • You do not manage quality, and don’t build quality control or QA activities into the schedule
    • You have more testing errors and more rework required
  • The client wants it done as quickly as possible
  • Your manager wants it done as quickly as possible
  • You think it can be done quickly, IF EVERYTHING GOES WELL
    • You commit to an estimate based on best-case
  • The client wants it done as quickly as possible
  • The client has a fixed amount for the budget
  • You think there is a chance you can get it done within available budget
    • You commit to an estimate based on available funding
  • The client tends to think the work is simple
  • The workers think they can do more than they can
  • The project manager does not take all the intangibles into account (estimating error, risks, problems, etc.)
    • You underestimate the work
  • You were burned on a similar project
  • You really don’t want to do the work
  • You tend to be pessimistic to begin with
    • You overestimate the work and hope it gets cancelled
  • Understanding common errors and not repeating them leads to more accurate estimates
• Estimating Effort, Duration and Cost
  • Estimate effort hours first
  • Duration estimates are next, based on known and unknown resources
  • Cost estimates are created after the rest of the schedule is completed
• The Estimate is Too High
  • The estimates are not arbitrary
  • Ask for more time to do more accurate and detailed estimate
  • You must balance the quality (effort), cost (budget), and time (schedule)
  • Look for ways to reduce costs using same effort
    • Less expensive resources
    • Older equipment
    • Internal instead of external training
  • Look for ways to reduce effort while still delivering
    • JAD sessions for requirements
    • Outsourcing
  • Look for ways to reduce the scope of work
    • Cut out less important reqs
    • Defer to later iteration
• Best Defense – Preparation
  • The Estimating Packet
    • You understanding of the work
    • The process used to prepare the estimate
    • The estimating technique(s) you used
    • A summary of the estimate
      • Effort, duration and cost as appropriate
    • The level of uncertainty
      • Estimating contingency
    • The estimating assumptions
    • The detailed estimating information
      • An appendix

Presenter – Tim Peek

• History of lifecycle development (mid-70s)
  • Waterfall methodology
    • BDUF – Big Design Up Front
  • Iterative methodology
    • Phase based development
    • Analysis Paralysis is a risk – mitigated by timebox or other constraints
• Agile Philosophy
  • Attempts to look at the root causes of software project failures
    • Good people handicapped by bad processes
    • If value is in the working system, then too much time is spent on non-value-added work
    • Iterative development is valid model but still overburdened by processes
  • “Let programmers be programmers”, not project managers
  • Counterculture swinging against heavily process-based methodologies
    • PMI and PMBOK (Project Management Body of Knowledge)
    • Capability Maturity Model (CMMI)
  • Early proponents appeared in the 1980s
    • Rapid Application Development
  • New ideas, visibility and momentum in the 90s
  • “Agile Manifesto” in 2001 started to bring Agile into the mainstream
• Manifesto
  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan
  • Early and continuous delivery of valuable software
  • Welcome changing requirements EVEN LATE IN DEVELOPMENT
  • Deliver working software frequently
  • Business people and developers work together daily throughout the project
  • Build projects around motivated individuals
    • Environment
    • Support
    • Trust
  • Communication face-to-face conversation
  • Working software is the primary measure of progress
  • Agile processes promote sustainable development
  • Continuous attention to technical excellence and good design
    • Continually developing AND
    • Continually testing – testing is key
  • Simplicity (maximizing the work NOT done) is essential
  • The best architectures, requirements and designs emerge from self-organizing teams
  • At regular intervals, the time reflects on how to become more effective, then tunes and adjusts its behavior accordingly
• Initial Agile Mindset
  • Focus on execution
    • Planning is important by not as important as execution
  • Writing plans and documents for development is wasting time from actually developing software
• The Current Agile Mindset
  • Focus on leaner versions
  • Agile project management
  • Agile quality assurance
  • Agile CMM
• The More Mature Agile Mindset
  • The software development world has moved too far toward process-focus
  • Too many processes result in bureaucracy and overhead
  • Most technical people work fine if you allow them
  • Things are rarely black and white
    • Make things as simple as possible, but not chaotic
    • Compromise when necessary
• The Nature of Projects
  • Traditional PM assumes projects are predictable and most are not
  • The nature of the project should be uncovered through exploration
• The Nature of Agile Projects
  • Generally software development
  • Smaller teams (3-5 people)
  • Larger projects
    • Split into smaller Agile projects
    • Use common processes and techniques
    • Might need longer sprints to manage coordination
    • Requires more structure and coordination
      • but keep minimal
• The “Planning Game”
  • Customer, management, users and team meet to determine functionality for next Sprint
  • The team explores requirement details with the customer
    • Detailed specs
    • Detailed test rules
  • Tech team meets afterwards to determine how to meet expectations
  • Ultimately, customer and project team agree what to work on
• Customer Requirements
  • Create test cases along with the requirements
  • Test cases validate completeness and correctness
  • Try to identify means for automated testing
• Project End Date
  • Customer prioritizes
  • High value requirements delivered first
  • Ultimately customer decides if the value is more than the cost of the next sprint
  • If not, the project will end
• Fixed Budget/Deadline
  • Budget delivers sprints with most valuable features until the money runs out
  • Deadline delivers sprints with most valuable features until the time runs out
  • Unlike traditional development, customer will ALWAYS have something delivered (fully tested production code)
• Construct
  • Refactor
    • Look for opportunties to enhance code
    • Make code more adaptable
    • Maintainability is key
    • Need to have solid regression testing process to ensure nothing breaks
    • Don’t try to make it perfect, just look for opportunities to make it better
• TDD
  • Test are defined along with detailed requirement
  • Write the test first, then code to meet the test
  • New tests get added to the repository
  • Programmer tests for 100% of the code
  • Tests must ALL run correctly with each sprint
  • Be sure that tests don’t all become too discrete.  Keep eye on overall design and create tests for integration and end-to-end logic
• Implementation
  • Continuous Integration
    • Daily builds ensure full integration
    • Uncovers integration bugs early, eliminates code freezes
  • Customer decides when the first iteration is usable and then grow features from there
• Scrum
  • “Scrum” metaphor used in 1986 to describe new approach to product development
  • Rugby corollary – adaptive, quick, self-organizing, few rests
  • Scrum approach developed and refined in 1996
  • Looks at software development in “new product” versus “manufactured product” model
  • Scrum Master
    • Management rep
  • Scrum teams
  • Daily Scrum – standup meeting every day
• Extreme Programming (XP)
  • Communication
  • Feedback
  • Whole Team
  • Planning Game
  • Customer Acceptance Tests
  • Small releases
  • Pair Programming
  • TDD
  • Design Improvement/Refactoring
  • Continuous Integration
  • Collective Code Ownership

Presenter – Jeannette Cabanis-Brewin (Center for Business Practices)

• The Big Debate
  • Nature or Nurture?
  • Experience or education?
  • Hard skills or soft skills?
  • The carrot or the stick?  Is it the organization or is it the person?
• Aspects of Project Manager Competency
  • Knowledge
  • Skills
  • Experience
  • Personal Qualities – find out who is best and see what they do
  • The higher you rise in an organization, the less you use technical skills and the more you use facilitative skills
• People make better project managers
  • When they focus on
    • Listening
    • Communicating
    • Collaborating
    • Being sensitive to the environment
• What research tells us
  • Culture is the software of the mind
    • Programmed, not hardwired
    • Reprogrammable
  • Most problems on projects are not technical problems, they’re “people problems”
    • Communications
    • Politics
    • Trust
• Beyond Either/Or (there’s “Both/And”)
  • Experience AND Education
  • Nature AND Nurture
  • Hard skills AND Soft Skills
  • Personal Competence AND the organizational structures that nurture and develop it.
• Traits of best-practice Project Managers
  • Recognized by stakeholders as the most important factor
  • Truthful
  • Exhibit eagerness to organize and lead
  • Personal confidence
• What to look for in a PM
  • Conflict Management Skills
  • Decision-Making Skills
  • Communication Skills
  • Intrinsic Motivation
  • A “Process Orientation”
  • Thinking Skills – Critical and Analytical Thinking, as well as being a Connected Knower
  • Integrity
  • Initiative
  • Perseverance
  • Humility
• Seven Habits for a good PM (See Steven Covey)
  • Be Proactive
  • Begin with the End in Mind
  • Think Win/Win
  • Put First Things First
  • Seek First to Understand
  • Synergize
  • Sharpen the Saw
  • Never Stop Learning
• Organizational Competence
  • Organizational culture can make or break good project managers
    • Supportive executives
    • An organizational home – the PMO
    • Project-centered role design
  • Fork in the Road strategy
    • Split the project manager role
      • Art
        • Communication
        • Analysis
        • Methodology
      • Science
        • Software
        • Technical
    • Use a competency model to determine best candidates for both

Presenter – Jeff McWherter(Web Ascender)

Why testing?  Peer Pressure – The secret to success is knowing who to blame for your failures.

Tools

• Naming Convention – testing for behaviors, not actual functions
• TestDriven.NET
• Free for non-commercial
• NUnit – Green Test – test passes (Red, Green, Refactor)
• Free/Open Source
• MBUnit
• Free/Open Source
• Uses RowTest to allow it to pass multiple Rows for testing Exceptions, etc.
• MSTest – tends to be slow
• Resharper – Helps with TDD
• VSTS – very expensive, simple to use
• Allows for finding text
• Regex results testing
• Fiddler – provides http/tcp/ip information
• GZip compression, performance improvement modeling
• Simulate modem speeds
• Firebug – provides bug checking
• Plugin – YSlow Plugin – grades app and shows performance (put JS at the bottom)
• Check viewstate
• Watin (inspired from Water framework) free functional web testing
• Create a browser object
• Navigate to website, click something
• Expect something
• IE and Firefox testing
• Test Recorder
• Selenium – Web Test Recorder
• Suite of tools including remote control, writes test code for multiple languages
• WCAT – Script based performance testing
• ANTS Profiler – Code profiling

Creating good tests

• Use your Right BICEP
• Right results
• Boundary using conditions CORRECT
• Inverse relationships check
• Cross-check results
• Error conditions can be forced to happen
• Performance characteristics within bounds
• CORRECT
• Conformance – does the code conform to an expected format?
• Ordering – is the set of values ordered or unordered as appropriate?
• Range – is the value within reasonable minimum and maximum values?
• Reference -
• Existence
• Cardinality
• Time
• Use Stopwatch to time class run
• Good tests are “A TRIP”
• Automatic
• Thorough
• Repeatable
• Independent
• Professional

Test Driven Development

• Concept is red, green, refactor
• Write test first, have it fail
• Write code to make it pass
• Refactor code as necessary

Behavior Driven Development

• For a given [something], do [something], should return [something]

Testing For Performance

• Performance Counters
• Memory pages/sec & available bytes
• Processor Time
• SQL Server Buffer Cache Hit Ratio
• SQL Server User Connections
• .NET CLR Exceptions thrown
• ASP Applications App_Requests_Total

Books for Test-Driven Development – Ken Beck

 

Presenter – Tim Peek (TenStep)

• Process is more important than tools (“Any fool with a tool is still a fool.”)
• Requirements fill in the detail of your scope statement
  • Once requirements are approved, they become part of the scope.
  • Fuzzy scope leads to inaccurate/fuzzy requirements
• Process/functional requirements
  • How it “functions” or works
  • How the deliverable interacts with the user
  • What happens first, second, third
  • How exceptions are handled
  • One subprocess’ output is another’s input
  • If condition A happens, then event B occurs
• Non-functional requirements (features)
  • How it looks
  • Description
  • Size, color, strength
  • look and feel
  • Performance (speed, capacity, reliability…)
• False requirements
  • Opinions (“I think we’re spending too much money on this”)
  • Projected-related statements (“We should prototype this first.”)
  • Assumptions (“The new release will solve this problem.”)
  • Out of scope (“While we are discussing reports, look at shiny objects.”)
  • Technology decisions
  • “Required” aspects, but not “requirements”
    • Policies and standards
    • Technical Architecture
    • Constraints
    • Deadlines
    • “We have to use XP, etc.”
• Technique – use cases
  • Define features and functions
  • Defined in terms of scenarios
  • “A user does an action and something happens”
  • Can use the use case to test for requirements
• Technique – prototyping
  • Create a model of defined requirements
  • Not fully functioning
    • Quick and Dirty
    • Hardcoded Components
    • Minimal functionality
  • Provide users with a visual representation of current state of requirements
  • Forum to more clearly define requirements
  • Uncover “hidden” requirements
• Requirements Development
  • Elicitation – I ask, you talk, I listen (write it down)
  • Validation – I analyze, I ask follow-up questions
  • Specification – I document, I ask follow-up questions
  • Verification  - We all agree
• Techniques for elicitation
  • 1x1 interviews
  • Group interviews
  • Facilitated sessions
  • Joint Application Design Sessions
  • Questionnaires
  • Prototypes
  • Follow people around (Walk-a-mile)
• Validation
  • Where the “analysis” starts
  • Make sure requirements gathered from Elicitation are accurate
    • Consolidate
      • Put requirements for similar processes and deliverables together
      • Please all the requirements at the same level
      • Determine if you have gaps
      • Eliminate requirements that are out of scope
      • Highlight areas of disagreement in the requirements
    • Rationalize
      • Fill in the gaps
      • Reconcile disagreements
        • Majority Wins
        • Escalate
        • Look at potential implications
      • Eliminate non-requirements
      • Put a full picture of all potential requirements
    • Model (Optional)
      • In addition to textual requirements
      • Adds precision to the requirements
      • Uses a precise modeling syntax
      • Usually requires training to create models
      • May require client training to interpret
• Specification
  • Prioritize
    • Determine the scale
      • H, M, L
      • Must, need, nice to have
      • Numeric Scale (1-3, 1-5)
    • Get consensus on the ratings
      • Majority wins
      • Escalate
      • Look at potential implications
  • Ensure traceability
    • Assign a tracking indicator to all requirements
      • Testing
    • Assign tracking indicators to all design, construct and test components
    • Create cross references to track requirements through the lifecycle (Traceability Matrix)
  • Ensure testability
    • Validate that they can be tested
    • Ensures precision
    • Helps catch any statements that are not requirements
    • If it can’t be tested, it’s not valid
    • In some methodologies, build the logical test case now
  • Create Business Requirements Report
    • Major Deliverable of the Analysis Phase
    • Draft up until now
    • Requirements finalized
    • Be as concise as possible
    • Eliminate potential misinterpretation
• Conduct requirements review
  • Formal meeting for final review
  • Does NOT include all stakeholders
  • All interested stakeholders have already seen
  • Review with decision makers and sponsor
  • Review is typically for QA, not accuracy (should already be accurate)
• Obtain sponsor signoff
  • Written signoff is best
  • Email is Ok
  • Verbal is NOT acceptable
  • Approval by default NOT acceptable
  • If sponsor will not approve, project STOPS

The last few weeks, with my return to focus on development, have been somewhat revolutionary for us as a development team.  We’ve really started working on what we want to be as a development team, and where we want to go.  As a team, we read the excellent e-book Foundations of Programming by Karl Seguin, which is kind of a peek into the ALT.NET world. 

That caused some developer soul searching, as we realized that we wanted to focus on Domain Driven Design, but we’re not 100% convinced that NHibernate is the way we want to go for O/R Mapping.  Maybe I’m misunderstanding it (and that’s certainly a reasonable expectation), but in our case, where we are using MS SQL, period, it seems like a lot of decoupling for no real return.

In the context of YAGNI, it seems like a lot of extra mapping that doesn’t really need to be done, if one uses LINQ to SQL, as we do.  Now, granted, I recognize that does inherently somewhat limit all of the testing that can be done.  But if the focus is on the domain objects, my thought is that if they are tested thoroughly, and pass, then we’ve certainly done due diligence and should be good to go.

As much as I recognize that becoming a better developer by understanding and applying more OOP principles to our development is essential for career progression, longevity, and personal satisfaction, at the same time, I recognize that it can go too far.  I DON’T have a C background.  I DON’T have a Java background.  Shoot, I don’t have a programming background, outside of the .NET world, so I have grown up eating the Microsoft cereal, and drinking the Microsoft Kool-Aid, and I haven’t found myself to be malnourished or parched, so it must not be THAT bad, right?  Granted, I’d like to make that cereal into more of a 3-egg omelet, and I’d love for that Kool-Aid to become more like sparkling grape juice, which is why we’re working on becoming better OOP developers, but I certainly think that baby steps in that direction are the right move.

On another front, that of overall project management methodology, we had proven to us this week the necessity for moving towards Agile/Scrum.  We had a project which was not properly scoped out, with little communication and guess what?  It ended up not being what our “customers” wanted and required additional work out of scope to get it up to where it needed to be.  It was absolute solid and real proof that we HAVE to move to iterative development.   We have to include the project sponsors in weekly meetings to review progress and make small changes, rather than sweeping ones.

Now I’ve just got to find some good training for the team…

Over the past few weeks, I’ve woken up.  Stepping back out of an IT Director role to being the Development Team Lead has helped me really get focused on where this team needs to go, thinking wise.  We have, to this point, developed some really great apps that help make this agency work.  But there’s so much more we need to do, and we have to get some thinking in place regarding how we do things.

Luckily, in our new IT Director, we have a man with MUCH experience and wisdom who is getting Project Management principles established, and that allows me to focus on how we can code going forward to ensure we are as effective as possible.  Of course, I’ve decided to look into Agile, and the best way to apply principles of iterative development to this ASP.NET team.  Luckily, I ran across a GREAT e-Book about ALT.NET and changing the way of thinking as an ASP.NET developer.  That e-Book is called Foundations of Programming, written by Karl Seguin (at http://www.codebetter.com).  I’ve read the first 20 pages so far, and I’ve assigned it to the team to read in order to create some thinking.  It definitely did that for me.

Having started my true development career in ASP.NET at Dell, I essentially ended up doing everything the MSDN way (as Karl puts it).  As a result, I broke nearly all the rules of classic development.  I’m having to rethink the way I’m doing things now, in order to make them what they should be.

So why Agile?  Why use Test Driven Development?  To paraphrase Sir Winston Churchill, “Agile and TDD are the worst forms of development except for all those others that have been tried.”  It’s funny how reading and working on the code in a different way has been really challenging to the status quo.  Along those lines, John Maxwell quoted Ronald Reagan in Developing The Leaders Around You, as saying, “Status quo, you know, that is Latin for the mess we’re in.” [emphasis added].  Well, we’ve not been in a real mess, but it could certainly be better.  So, we begin thinking in terms of iterations, not huge builds.  We begin thinking in terms of YAGNI (You Ain’t (hey, I’m in the S’th <—Mitch Hedberg reference, btw) Gonna Need It).  We begin thinking in terms of decoupling the code from the UI.  For real.  We start looking at ASP.NET MVC, and we start to really get it all of a sudden.

To my friend Jayme Davis, huge thanks for providing a little bit of information to help reassure me that I’m not losing my mind here.  He was a mentor at Dell, and has continued to be a great mentor ever since.

It feels good to be awake…