qiaohua@taobao.

com
2010-10-14

UNDERSTANDING JAVASCRIPT
TESTING
Why?

 Cross-browser issues.
 跨浏览器问题 ;
 The possibility for causing an
unforeseen problem is simply too
great.
 不可预见的问题的存在性很大 ;
How?
Test strategy

- End-to-end test:
- Big, Powerful, Convincing;
- Slow, In-exact, High-maintenance;
- Unit tests
- Small, Quick, Focused, Resilient;
- Limited;
- Component tests
- the balance between many fast test
and a few slow tests;
Test Methods

- 测试脚本 + 模拟环境 ;
- 测试脚本 + 驱动真实浏览器 ;
- 直接在浏览器中 Unit test;
- 模拟环境下进行 Unit test;
Unit Testing

- Break code into logical chucks for

testing.
- 将整段代码分成多个逻辑块来测试 ;
- Focus on one method at a time
- 同一时间内只关注一个方法 ;

- 支持 UT 的已有工具 : QUnit, JSUnit, YUITest;

- Run now, run later;
- Run in new browsers;
- Put the test in a file rather
than Firebug;
Unit Testing Framework
Assertion Function
Tests/Test Case
- test('A test.', function(){
asset(true, 'something');
asset(false, 'something');
});
- setUp()/tearDown()/setUpPage()
- Async Tests;
setTimeout(function(){}, 100);
Test Suite
- addTestPage()/addTestSuite();
Test Runner
- Responsible for loading an executing tests;
Trace/log
- warn()/inform()/debug() 3 tracing levels;
传统单元测试 , 如 YUI3
Test Case
- 函数名组织： Classic + BDD;
- setUp / tearDown;
- should: ignore, error, fail;

Assertions
Mock Objects
Asynchronous Tests
- wait;
- resume;

Test Suites
Test Runner
Test Reporting
var testCase = new Y.Test.Case({

name: "TestCase Name",

testSpecialValues : function () {
Y.Assert.isFalse(false); //passes
Y.Assert.isTrue(true); //passes
Y.Assert.isNaN(NaN); //passes
Y.Assert.isNaN(5 / "5"); //passes
Y.Assert.isNotNaN(5); //passes
Y.Assert.isNull(null); //passes
Y.Assert.isNotNull(undefined); //passes
Y.Assert.isUndefined(undefined); //passes
Y.Assert.isNotUndefined(null); //passes

Y.Assert.isUndefined({}, "Value should be undefined."); //fails

}
});
Behavior Testing

- Similar to unit testing, but broken up by

task;
- Functionally very similar to unit testing,
uses different terminology;

- 支持 BT 的现有工具 : Screw.Unit , JSSpec,

Jasmine
如 : Jasmine

- code is specification;
- describe 即是 TestCase, 也是 TestSuite;
- ignore 更简单，加上 x 即可 ;
- Matchers 可自定义，可覆盖，可添加 ;
- toThrow 比 YUI Test 更易用 ;
- expect 本身就是一句描述，无需注释 ;
- Spies
describe('Calculator', function () { var testCase = new Y.Test.Case({
var counter = 0
name: "TestCase Name",

it('can add a number', function () { testSpecialValues : function () {

counter = counter + 2; // counter was Y.Assert.isFalse(false); //passes
0 before Y.Assert.isTrue(true); //passes
Y.Assert.isNaN(NaN); //passes
expect(bar).toEqual(2);
Y.Assert.isNaN(5 / "5"); //passes
}); Y.Assert.isNotNaN(5); //passes
Y.Assert.isNull(null); //passes
it('can multiply a number', function () { Y.Assert.isNotNull(undefined); //passes
Y.Assert.isUndefined(undefined);
counter = counter * 5; // counter was //passes
2 before Y.Assert.isNotUndefined(null); //passes
expect(bar).toEqual(10);
}); Y.Assert.isUndefined({}, "Value should
be undefined."); //fails
});
}
});
TDD vs BDD
- TDD is not about testing, but rather about design and
process;
- TDD is a design activity;

Why TDD?
- Makes you think about required behavior;
- Reduces speculative code;
- Provides documentation;
- Improves quality;
BDD

 BDD 的重点是通过与利益相关者的讨论取得对预期的软件行为的清醒认识。

 它通过用自然语言书写非程序员可读的测试用例扩展了测试驱动开发方法。

 行为驱动开发人员使用混合了领域中统一的语言的母语语言来描述他们的代码的目的。

 这让开发着得以把精力集中在代码应该怎么写，而不是技术细节上，

 而且也最大程度的减少了将代码编写者的技术语言与商业客户、用户、利益相关者、
项目管理者等的领域语言之间来回翻译的代价。
Jasmine 实战

 Specs:
 说明 , 使用 it(description, fn) 来描述 ;

 it('should increment a variable', function ()

{ // 一段有意义的描述 , 加一个要执行的系列动作
 var foo = 0;
 foo++;
 });
 Expecations:
 期望 , 存在于 spec 中 , 用来描述你期望得到的结果 , 使用
expect() + matchers;

 it('should increment a variable', function () {

 var foo = 0; // set up the world
 foo++; // call your application
code

 expect(foo).toEqual(1); // passes because

foo == 1
 });
 Suites
 Specs 的集合 , 等于 Test Case, 使用 describe() 函数 ;

 describe('Calculator', function () {
 it('can add a number', function () {
 ...
 });

 it('has multiply some numbers', function () {

 ...
 });
 });

 Suites 的名字一般为你要测试的模块 / 组件 / 应用名字 ;

 Suites 中的每个 Spec 只执行一次 , 一个 Suites, 一个作用域 , 里面的 Spec 共享 ;
 Nested Describes
 支持嵌套的 Describes;
 beforeEach(fn)/afterEach(fn) --- 对应于以前的
setUp(fn)/tearDown(fn) , 在每个 spec 执行之前 / 之后 执行 ;
 this.after(fn) 在特定的某个 spec 执行之后执行 . 没有 this.before !

 describe('some suite', function () {

 it(function () {
 var originalTitle = window.title;
 this.after(function() { window.title = originalTitle; });
 MyWindow.setTitle("new value");
 expect(window.title).toEqual("new value");
 });
 });

 xit()/xdescribe() 设置 spec/describe 不可用 .

 Matchers
 expect(x).toEqual(y); compares objects or primitives x and y and
passes if they are equivalent
 expect(x).toBe(y); compares objects or primitives x and y and
passes if they are the same object
 expect(x).toMatch(pattern); compares x to string or regular expression
pattern and passes if they match
 expect(x).toBeDefined(); passes if x is not undefined
 expect(x).toBeNull(); passes if x is null
 expect(x).toBeTruthy(); passes if x evaluates to true
 expect(x).toBeFalsy(); passes if x evaluates to false
 expect(x).toContain(y); passes if array or string x contains y
 expect(x).toBeLessThan(y); passes if x is less than y
 expect(x).toBeGreaterThan(y); passes if x is greater than y
 expect(fn).toThrow(e); passes if function fn throws exception e
when executed

 expect(x).not.toEqual(y); compares objects or primitives x and y and

passes if they are not equivalent
 Matcher 是可以自定义的 . 使用 addMatchers(obj)

 toBeLessThan: function(expected) {
 return this.actual < expected;
 };

 beforeEach(function() {
 this.addMatchers({
 toBeVisible: function() { return
this.actual.isVisible(); }
 });
 });
 Spies
 permit many spying, mocking, and faking behaviors.
 用于模拟传参 , 回调函数 , 异步请求 / 行为监测

 it('should spy on an instance method of a Klass', function() {

 var obj = new Klass();
 spyOn(obj, 'method');
 obj.method('foo argument');

 expect(obj.method).toHaveBeenCalledWith('foo argument');

 var obj2 = new Klass();

 spyOn(obj2, 'method');
 expect(obj2.method).not.toHaveBeenCalled();
 });
 Asynchronous Specs
 异步测试 , 测试 ajax api, 事件回调等 , 就是针对在未来某个点上会
发生的行为describe('Spreadsheet',
.it('should calculate thefunction() {
total asynchronously', function
 runs() 阻塞执行 , 就像是直接调用一样
var spreadsheet = new ;Spreadsheet();
多个 runs() 共享作用域 .
spreadsheet.fillWith(lotsOfFixureDataValues());
 waits(timeout) 等待多长时间后再执行下面的语句 .
spreadsheet.asynchronouslyCalculateTotal();
 waitsFor(function, optional message,
optionalwaitsFor(function()
timeout) 直到 {
function 返回 true 才执
return spreadsheet.calculationIsComplete();
行下去 . }, "Spreadsheet calculation never completed", 1000
runs(function () {
expect(spreadsheet.total).toEqual(123456);
});
});
});
 http://kissyteam.github.com/kissy/tes
ts/index.html
其他相关
Automation
- Functional Testing
- Selenium IDE:
- records and automates actions performed by a user;
- An extension for Firefox that records the actions;
- Can play them back in all browsers(limited by cross-
domain issues);
- Primarily for testing web applications, everyone should
use it;

- Browser launching
- WebDriver;
- Waitr;
- JsTestDriver;
- Selenium RC;
- Server-Side
- Ignore the browser! Simulate it on the server-side;
- Almost always uses Java + Rhino to construct a browser;
- Some frameworks
- Crosscheck: Pure Java, even simulates browser bugs;
- Env.js: Pure JavaScript, focuses on standards support;
- Blueridge: Env.js + Screw.Unit + Rhino;

- Distributed
- Selenium Grid
- Push Selenium tests out to many machines(that you manage),
simultaneously;
- Collect and store the results;
- TestSwarm
- Push tests to a distributed swarm of clients;
- results viewable on the server;
- testswarm.com;
The Scaling Problem
- All need to be run for every commit, patch, and
plugin;
- JavaScript testing doesn't scale well;

Distributed Testing
- Hub server;
- Clients connect and help run test;
- A simple Javascript client that can be run in all
browsers, including mobile browsers;
- TestSwarm;
JSTestDriver

- 服务端 / 客户端 ,
- test runner 捕获浏览器 , 通过命令通知服务器进行测试 .
然后每个被捕获的浏览器运行 tests, 并将结果返回 ;
- 优点 :
- 运行测试不需要手工跟浏览器进行交互 ;
- 可在多台机器上运行 , 包括移动设备 , 允许任意复杂的测试 ;
- 测试非常快速 , 因为不需要操作 DOM, 且多个浏览器中是同时进行 ;
- 现已支持异步请求测试 ;

- 缺点 :
- JavaScript required to run tests is
slightly more advanced, and may cause a
problem in old browsers;
使用 JSTestDriver

目录结构 :
JSTestDriver
- jsTestDriver.conf # 配置文件
- JsTestDriver-1.2.2.jar # 核心程序 , 包含客户端 / 服务器
- src/ # 待测试 js 源码
- src-test/ # js 测试脚本

配置 :
server: http://localhost:9876

load:
- src/*.js
- src-test/*.js
服务器 : java -jar -jar
D:\workspace\Test>java JsTestDriver-1.2.2.jar --
JsTestDriver-1.2.2.jar --tests all --verbose
[PASSED] cookie get.test that it should return the cookie value for the
port
given na 9876
me
浏览器捕获 : http://localhost:9876/capture
[PASSED] cookie get.test that it should return undefined for non-
existing name
运行测试
[PASSED]:cookie
java -jarthatJsTestDriver-1.2.2.jar
set.test it should set a cookie with a given name --
and value
tests all
[PASSED] cookie remove.test that it should remove a cookie from the
machine
[PASSED] json stringify.test that it should convert an arbitrary value to a
JSON
string representation
[PASSED] json parse.test that it should parse a JSON string to the native
JavaSc
ript representation
Total 6 tests (Passed: 6; Fails: 0; Errors: 0) (0.00 ms)
Firefox 3.6.10 Windows: Run 6 tests (Passed: 6; Fails: 0; Errors 0) (0.00
结合 jasmine

更改配置为 :
server: http://localhost:9876

load:
- ../github/new/kissy/tests/jasmine/jasmine.js
<-----
- ../github/jasmine-jstd-adapter/src/JasmineAdapter.js
<-----
- ../github/new/kissy/src/kissy/*.js
- ../github/new/kissy/src/cookie/cookie.js
- ../github/new/kissy/src/cookie/tests/cookie.js
IDE 中使用

 IDEA 安装 JSTestDriver plugin, 重启

IDEA , 就可以看到
 jstestdriver.gif
 cmd 下 , java -jar JsTestDriver-1.2.2.jar
--tests all
小结一下
TestSwarm 众包测试

TestSwarm provides distributed continuous

integration testing for JavaScript.

why? -- JavaScript Testing Does Not Scale

The primary goal of TestSwarm is to take the

complicated, and time-consuming, process of
running JavaScript test suites in multiple
browsers and to grossly simplify it. It achieves
this goal by providing all the tools necessary for
creating a continuous integration workflow for
your JavaScript project.
中心服务器 , 客户端连接至他 , job 提交到这里 ;

客户端是一个 test runner 实例 , 加载在浏览器中 .

test runner 每 30 秒中请求服务器是否有新的 test suites 需要运行 , 如果有 , 就执行 ( 放

在一个 iframe 中 ), 其结果发送到服务器上 . 没有就睡眠等待 ;

一个 job 包含 test suites 和 browsers( 需要在哪些浏览器中进行测试 ), 运行至少一次 .

私有成员测试
Approach 1: Don't Test Private Methods
- 如果你需要对私有成员做测试时 , 那就应该要考虑是否将它转成公有方法 ;
- 间接测试 , 测试那些调用该私有成员的公有方法 ;

Approach 2: Give the methods package access.

- 给私有方法套层 package;
- but it does come with a slight cost.

Approach 3: Use a nested test class.

- to nest a static test class inside the production class being
tested.
- how?

Approach 4: Use reflection.

- it provides a clean separation of test code and production code.
UI 测试
① DOM
② 事件模拟

目前提供 Web UI 测试的工具 : record -> play

- Selenium
- Selenium is a robust set of tools that supports rapid development
of test automation for web-based applications.
- Selenium provides a rich set of testing functions specifically geared
to the needs of testing of a web application.
- Watir
- It allows you to write tests that are easy to read and maintain. It is
simple and flexible.
- Watir drives browsers the same way people do. It clicks links, fills
in forms, presses buttons. Watir also checks results, such as
whether expected text appears on the page.
- SIKULI
展望
- 全自动化
- WebUI Test Studio 功能强大的集成开发环境
- Testcase Management, Execution, and Source Control;
- Integration with Visual Studio Unit Testing;
- Powerful Automated Test Recorder;
- DOM Explorer;
- Point-and-click Test Recording Surface;

- 自动报告的生成

- 众包测试

- 全网测试
Thanks for your
attention!