You are currently browsing articles tagged community.

This is the second half of our two-part discussion on crowdsourcing with uTest’s VP of Marketing and Community, Matt Johnston.  The first installment of our interview is here.

So, you’ve gathered a bunch of information and data from your users.  How do you address issues of IP and who owns the content, both from the perspective of your client companies and from your users?  How do you protect the rights of both of those groups?  Do either of those groups give certain aspects up by using your model that they may not have to otherwise?

That’s a legitimate issue for crowdsourcing companies who are creating something new – for example, in categories like copy writing, design, development and animation.  In such instances, companies like Guru, crowdSPRING or GeniusRocket have to think carefully about fairness and address these issues head on and up front.  Clear communication (and consistent enforcement) of the rules is vital to maintaining a level playing field in a B2B crowdsourcing model.

In the case of uTest, we don’t have to worry about issues of IP ownership because the customer created the apps and simply wants to test them with our community.  The IP issues that are sometimes a concern for new customers are around IP protection.

Again, in this case our job is to establish the ground rules that protect everyone, and then to communicate them clearly and consistently.  In the case of uTest, the client created the app and they own it.  Testers have no rights to the app or to any of the bugs that they discovered within that app.  Beyond that, companies can require testers to sign an NDA and testers are forbidden from sharing any information about customers, their apps, or any bugs outside the uTest platform (including blogs, message boards, Twitter, Facebook, etc.).

You’ve spoken about 90-9-1 Rule, which says that 90% of your community will be passive users, 10% will be active and engaged, and 1% will be the “stars.”  Can you shed some light on these numbers and how each group can add value to the process?

We had a great discussion about this at the recent TiE event!  The 90-9-1 rule states that 90% of community members will be “lurkers,” 9% will be part-time contributors and 1% will fully adopt and become “super users.”  While the math changes from crowd to crowd, the underlying premise does not:  the vast majority of the contribution or participation will come from your top users.  This isn’t just true in crowdsourcing, it’s true on message boards, social networks, blogging services, and most online user bases.

If you’re building an ad-based revenue model, you may not care, because even “lurkers” generate page views.   However, if you’re building a transaction-oriented revenue model like uTest is, you need participation in order for a community member to make a direct contribution.

Thus, one of uTest’s keys to success is to identify those future “stars” early on so we can nurture them along the path to success.  We recognize that some testers join our community because they want to use their testing skills to earn money, while others join because they want to network with their peers and consume the educational content that we provide.  We welcome all testers into our community, but we make a special effort to engage those who want to participate in projects.

We talk a lot about “inside-the-transaction” vs. “outside-the-transaction” types of participation.   The former refers to testers who are participating in projects, reporting bugs and getting paid (as well as building their reputation).  The latter refers to those who don’t participate in projects, but could still make meaningful contributions.  This could be by subscribing to our newsletter, commenting on our blog, participating in our forums, or contributing content, ideas and expertise to other members of our community.  Both types of participation are valuable to uTest’s long-term future, but the wants and needs of these two groups are unique, so we cater to them as such.

Your company didn’t just come to the decision to implement crowdsourcing one day, you actually built the entire company on that model specifically.  It’s in the name - uTest.  Do you feel you could have gotten similar results had you been offering the same end-products through more traditional methods of production/testing?  Would they have been worse?  Could things be better using a different model?

If we tried to do this with a traditional model, the results wouldn’t even be close to what we’ve achieved to date.  It would be impossible from a logistics or financial perspective for us to have built the level of testing coverage that we have today.  We’re doing testing for some of the top software companies in the world in our first year of operations – no small feat.  And we couldn’t do it without our community and our investment in our community.

The testing needs in the world of software have become exceedingly complex. Companies now have to test their apps across locations, languages, operating system, browser, as well as handset makers and models, and wireless carriers.  This is a prohibitively expensive task for even the most mature and sophisticated companies.  And crowdsourcing is uniquely suited to meet the challenges of software testing.  That underlying belief is what prompted our co-founders to build uTest based on a crowdsourcing model – and what led me to join the company.

About uTest

uTest is the world’s largest marketplace for software testing services. The company provides real-world testing services through its community of 20,000+ professional testers from 158 countries around the world. Hundreds of companies - from web startups to global software leaders - have joined the uTest marketplace to get their web, desktop and mobile applications tested. More information can be found at or the company’s Software Testing Blog at

We’ve interviewed Matt Johnston, VP of Marketing and Community at uTest, to discuss some of the concepts around the use of crowdsourcing.  This is the first installment of that discussion.

uTest provides software testing services for web, desktop and mobile apps.  The company offers this testing to companies of all sizes – from five-person startups to Fortune 500 enterprises – by leveraging its community of 20,000+ QA professionals from 157 countries around the globe.

You have a pretty well-defined explanation of what crowdsourcing is to you - and what it isn’t.  In fact, I think you said that you had two big criteria that have to met for a process to be considered crowdsourcing.  Can you talk a bit about what this definition includes?  What doesn’t it include?

That’s a good question – and one that’s hotly debated in crowdsourcing circles.  From my perspective, the first criteria is that something of substance needs to be sourced. I’ve heard people say that they’re going to “crowdsource a question” and then simply publish a one-time poll with a single question – that’s not crowdsourcing to me, it’s glomming onto a media-friendly term.

Whether it’s for professional services like copywriting, photography, design, development or software testing, I think there’s some minimum level of contribution and value-add that’s required before you can say that something has been “sourced” (whether it be in-sourced, outsourced or crowdsourced).  I’m not trying to define what this minimum criteria is, but I know that there’s a line.

The second criteria is that the output comes from a “crowd”, not just a single individual.  There are many outstanding companies that use “onesourcing” models to great success (eg: Elance, Guru, OnForce).  And in many categories, what the customer wants may be to find that one person who precisely matches their needs, as opposed to a team of people or collaborators.  I simply don’t consider that crowdsourcing.  In other categories and companies, the deliverable requires a team of people making a contribution (eg: uTest, Trada) and the overall deliverable is the sum of the contributions.

When it comes to implementing the crowdsourcing model, there are a lot of things to think about: control issues, organization, data gathering etc.  What were some of the things uTest felt were the most important to address and how did you address them?

That’s the right question, because a common misconception is that crowdsourcing is easy… just build a crowd, add water, and watch the revenues roll in!   A few hard-earned, hard-learned lessons that I’ve picked up:

  1. This will sound odd coming from a marketing guy, but the first key to building a successful crowdsourcing model is structured data.  The truth is that it’s not too tough to build a loosely assembled, unstructured crowd – what we call a “mob”.  uTest’s plan, however, was to advance far beyond that and to create a rated, ranked community of professional testers.   We use structured data to ensure that we can precisely match each customer’s project with the right testers, based upon location, languages spoken, hardware, software, expertise and, above all else, past performance.  If you’re thinking about a crowdsourced model, structured data is the foundation upon which you’ll build it.
  2. Recognize that you have two sets of customers – in our case, we have software companies and we have our community of testers. Most businesses are hardwired to think of their customer base as their lone stakeholders.  The other constituencies (like vendors, employees, etc.) are there to serve the customer.  In a crowdsourcing model, you must realize that you have two distinct sets of customers to whom you need to cater.  Serving both audiences well on an ongoing basis is a difficult, but wholly necessary, balancing act.
  3. There are two types of currency in an online community: money and reputation. Of course, money serves as a primary motivator, but the transparency and accountability of crowdsourcing motivates community members to build strong reputations. The net effect is that top performers get promoted and poor performers get demoted.  In both the near- and long-term, this means that you’re simultaneously rewarding your best community members and better serving your clients.
  4. The final lesson is that crowdsourcing is a model – an approach to solving a problem.  It’s not a solution in and of itself.  There are some verticals where it’s a great fit, and others where it’s not.  If you’re considering incorporating crowdsourcing into your existing business (or launching a new business based upon this model), make sure that the space and the problem you’re solving are a good fit.

Management of the crowdsourcing model can seem - well - unwieldy and chaotic to people thinking about starting it out.  How have you handled this aspect of the process?  What are some key things to do or not to do?

This question made me laugh, because I was just debating this with a friend who runs a crowdsourcing company on the west coast.  Ultimately your clients aren’t coming to you because they want crowdsourcing (really, they’re not).  They’re coming to you because they have a problem to be solved and the traditional, status quo solutions aren’t cutting it and they’re exploring alternatives.

When it comes to managing crowdsourcing solutions, it can seem unfamiliar to clients.  In those cases, it’s critical that the crowdsourcing company offers services and tools to help clients climb the learning curve.  In cases like these, you’re really competing with muscle memory of the client (“I’m used to doing things in a certain way and vendors need to adapt to my needs …”).

Here are the steps that I’d follow if I was thinking about creating a crowdsourcing company or offering:

  1. Start by looking carefully at the problem you’re trying to solve and the audience for whom you’re trying to solve it.
  2. Understand how they’ve historically solved this problem.
  3. Discover what works and doesn’t work for them with these traditional solutions
  4. At this point, you have a decision to make:
    1. Reduce the management burden on your customers to the point that using crowdsourcing is no more time-consuming than managing the traditional solutions
    2. Convince the world that your crowdsourcing solution is vastly more effective (aka: better) or efficient (aka: cheaper) to the point that it’s worthwhile for them to learn a new motion.

More specifically, uTest has identified and leveraged the “reputation” system in their model.  How have you done that and how has it worked to your advantage in having an effective community?

The reputation system behind a crowdsourcing model – or any community/marketplace model – is critically important.  As stated previously, there are two types of currency in a healthy crowdsourcing model:  money and reputation.

Of course, money serves as a primary motivator, but a well-designed reputation system creates effective incentives (and disincentives) for community behavior.  Among a community of peers, how one is perceived is a powerful driver.  For example, if everyone can see that I’ve achieved silver status, then I’m proud of my accomplishments AND I’m driven to achieve gold status.

As for the particulars of a reputation system (invisible vs. visible, stars vs. points, etc), there is no single right answer.  Ultimately, it depends upon the space that you occupy and the make-up of your community.  For example, eBay reveals the reputations of buyers and sellers in order to promote good behavior.  Conversely, Google’s search algorithm (which is really just a reputation system for websites that tries to match the right sites with each query) is kept secret in order to prevent people from gaming the system.

The real key is to align your incentives (both monetary and reputation) with the desired behaviors.  Before we ever build a new feature or make a change in our policies or pricing, we talk a great deal about what behavior are we incenting or disincenting.

We’ll be posting up the second half of our interview in the coming days, so be sure to check back!  Do you have any experiences in crowdsourcing?

[Update: The second half of our interview is up]

About uTest

uTest is the world’s largest marketplace for software testing services. The company provides real-world testing services through its community of 20,000+ professional testers from 158 countries around the world. Hundreds of companies - from web startups to global software leaders - have joined the uTest marketplace to get their web, desktop and mobile applications tested. More information can be found at or the company’s Software Testing Blog at

Anya and I attended the WorldRG-sponsored Business of Community Networking conference in Boston last week.  The setting was very intimate, and there was a great line-up of speakers.  I’ve given a run down of several of them below, with some of the main takeaways.

Clara Shih on her thoughts on Facebook and Online Communities

Panel on how various online communities and community platforms have worked or not worked

Lena West on Viral Marketing

Liz Strauss on Successful Blogging

Michael Cayley on Social Capital Value Add

A panel discussion on ROI measurement

Susan Getgood on Social Media and Customer Service

Marketing, Branding and Community: How social networks are rewriting the rules of marketing, branding and community

Clara Shih, author of The Facebook Era

  • Facebook is a way for humans to interact with each other, it’s not just a tool.
  • How can collaboration/productivity tools be incorporated into Facebook? Can it be used as a CRM?
  • It’s now become a social norm to share personal information publicly.  Facebook can be used as a channel to access information via “trusted online identity.”  You can connect with friends about what’s important to you, both personally and professionally.
  • How do you as a company insert yourself in the conversation in a way that’s valuable and not invasive?
    • Know your customer: use transitive trust, a personalized interaction.  It’s up to the individual to share their information.  Customers expect that you know them and that you’ve done your due diligence on them personally.
    • Weak ties are very important in this setting, leverage them.
  • Facebook offers a personal contact database.  A traditional CRM is uni-directional (companies push), now it’s bi-directional (the customer is empowered).

[Faceconnector demo]

  • There is a loyalty magnification effect in Facebook: Passive word-of-mouth (you can become a fan of something right from your own newsfeed.  If one of your friends becomes a fan, it’s very easy to follow suit).
  • Facebook offers precision marketing with hypertargeted ads
    • You can minimize wasted ads
    • Leverage latent interest
    • Test new segments and messaging
  • How do you reach them before intention sets in and get them to become intention-based buyers?

[Resources: You can install Faceconnect here on the apps page of and learn more about The Facebook Era here.]

Learnings from a Facebook Group in Business Investigation

Jenny Ambrozek, Victoria Axelrod, Francois Gossieux

  • You can now tap communities once only reserved for companies with deep pockets
  • Community development/management has to be considered a real investment
  • Facebook isn’t great for managing huge groups
  • Ning provides a platform for rich conversation (discussion threads, blogs, subgroups, etc)
  • Fundamentals
    • Good content
    • Allow for members profiles
    • Don’t think market segments, think tribes
    • Think about behavior, not demographics
    • Don’t think of the tool as a channel, think about it as a conversation between you and them
    • Content must be picked up to become part of the conversations
  • “Whether there’s an ROI or not, [social networking] is something you have to do.”
  • “We haven’t been able to assign a dollar amount to [social networking], but you still have to play.”
  • What is the relationship of new people coming to the site to those current members?  How did they get there?
    • Known name
    • Known friend of a friend
    • New face
    • New member, source unknown
  • Align your activity measurements with network measurements and analysis
  • It’s about behavior, not attributes
  • Reciprocity in people is a reflex
  • We either behave in a market framework (contract, employment, cold, calculated) or a social framework
  • Provide structure and house rules (top-down) while nurturing the bottom-up interactions
  • Facebook ends up being a lot of work
    • Much harder for B2B to get companies to interact
    • People don’t want to necessarily go to Facebook to do business
    • It’s still difficult to put a “face” to a company
  • Find out where your trive hangs out, go there, and deliver results

[Resources: Tribalization of Business Study (Beeline Labs)]

The Chicken or the Egg: The real deal about “viral marketing”

Lena West, Founder & CEO of Xyno Media

  • Viral Marketing is any marketing tactic/content that encourages “pass along” sharing, which then changes that messages level of influence.
  • 3-7-3 Frameworks
    • 3 Rules
      • Viral marketing is created, not born
      • People hate the term viral marketing, and probably always will
      • Not all viral marketing is good
    • 7 Criteria
      • Free & short rule
      • Doesn’t force behavior change, but allows for it
      • It’s not just entertaining, but people can see themselves doing it
      • Feeds off how people work
      • Scalability is hardwired (support structure needs to be there)
      • Facilitates easy sharing
      • Leverage Other Peoples’ Social (OPS)
    • 3 Imperatives
      • Listen/Monitor (if you don’t listen, you don’t know what’s going on)
      • Set the kill switch (how can you pull the plug if you need to)
      • Once your campaign goes viral, it no longer belongs to your brand

Understanding the Conversation Online Between Consumers: Focusing on blogging

Liz Strauss, Social Media Strategist and blogger at Successful Blog

Please see separate post complete with video for this presentation.

Social Media Reality: Achieving cultural shifts

Michael Cayley, Founder of Social Capital Value Add

[Video coming soon]


  • Two main points of this video:
    • We’re going through exponential change
    • Bandwidth is one of the key drivers
  • What’s In It For Me (WWIF Me) has become WIIF Them
  • There is an authentic connection and self-fulfillment found through organization
  • Would some companies survive if they weren’t aligned with CSR?
  • Shared perception is mediated
  • The medium is in the message
  • The scale of human beings has changed, so how we architect around that will change
  • Brand Valuation is an estimation of the future earnings of products and services
  • Social Capital in not based on a product line

[Resources: Introducing Social Capital Value Add e-book]

Understanding the ROI with Community Marketing

Chris Carfi, CEO of Cerado (Moderator); Myles Bristowe, President of Boston American Marketing Association and CMO of Commonwealth Creative Associates; Michael Cayley, Founder of Social Capital Value Add; Jenny Ambrozek, Founder of SageNet; Erica Farthing, Director of Social Media for

[Video coming soon]

  • Anything in marketing is a risk
  • Measure everything you can
  • You can measure so much more now than you could
  • Give your members a reason to join your network (for AMA is was to communicate with professionals in their field and get relevant information)
  • Instead of just having events or a newsletter, an online community offers value from the association or company every day
  • For AMA, in order to convert people from community members to Association members, there needed to be someone who reached out to them, they needed to find continual value, and they needed to participate in order to convert to membership.
  • Integrate the back end of your community for data capture
  • Most measurements are happening ad hoc, but creating a company picture with the most applicable measurements is key
  • Set realistic goals

Examining Social Media & Customer Services

Susan Getgood, Principal of Getgood Strategic Marketing

  • Your starting point is your customer
  • Customers are online talking about you
  • It’s about the social part.  The tool is just a medium and they’re changing every day
  • Public social netowkrs are where discussions are taking place
  • 85% of social media users say that companies should be online in a social networking way
  • It’s not just about outbound marketing, it’s about engagement and what the customers do once they reach you
  • 4 Ps of online engagement: Prepare, Participate, Pitch or Publish
  • Constistency, Honesty and Value

Bad Behavior has blocked 436 access attempts in the last 7 days.