Compare commits

...

456 Commits

Author SHA1 Message Date
Jakob Borg
c519e582b5 Expand tilde on Windows as well (fixes #289) 2014-05-26 16:58:03 +02:00
Jakob Borg
6b9dce36bf Default listen host should be 0.0.0.0 (again) (ref #216) 2014-05-26 15:01:04 +02:00
Jakob Borg
8e0520887a Send default permissions 0777 on directories when IgnorePerms set (ref #284) 2014-05-26 11:09:35 +02:00
Jakob Borg
cfd1fdb38e Don't set permissions 000 on directories with NoPermissionBits set (ref #284) 2014-05-26 11:08:54 +02:00
Jakob Borg
c6ba0208d0 Don't require SSE in 32 bit builds (fixes #277) 2014-05-25 21:36:38 +02:00
Jakob Borg
3d055bbb79 Simple file versioning (fixes #218) 2014-05-25 20:49:08 +02:00
Jakob Borg
dd971b56e5 Correct tests for uppercase-only node IDs 2014-05-25 14:54:50 +02:00
Jakob Borg
4031f5e24b Fix version comparison in upgrade 2014-05-24 23:22:08 +02:00
Jakob Borg
1cd7cc6869 Configuration directory is machine local (Windows) 2014-05-24 22:45:50 +02:00
Jakob Borg
9de2864db3 Repair and clean HTML structure 2014-05-24 21:56:09 +02:00
Jakob Borg
c27861cbaf Show node ID/name/address mapping at startup (ref #249) 2014-05-24 21:39:08 +02:00
Jakob Borg
c2f75d3689 Show counters for total data transferred (fixes #265) 2014-05-24 21:34:11 +02:00
Jakob Borg
5454ca1cf7 Sort list of sharing nodes (fixes #266) 2014-05-24 21:13:35 +02:00
Jakob Borg
8644bf30a9 Syncthing might be restarted after shutdown (fixes #274) 2014-05-24 21:08:53 +02:00
Jakob Borg
db3341a178 In Sync is now Up to Date (fixes #268) 2014-05-24 21:06:46 +02:00
Jakob Borg
e2cb0219c7 Node IDs are always upper case (ref #269) 2014-05-24 21:01:21 +02:00
Jakob Borg
217f29de76 Don't mess up unset properties of new nodes/repos 2014-05-24 21:00:47 +02:00
Jakob Borg
8661afcb4f Expand ~/ on Windows as well 2014-05-24 13:34:40 +02:00
Jakob Borg
ed07fc0f2c Simplify node/repo headers on extra-small screens 2014-05-24 12:38:44 +02:00
Jakob Borg
4af3f77a9a Wait for parent to release sockets (fixes #267, fixes #241) 2014-05-24 12:28:36 +02:00
Jakob Borg
8c4f07ef1b Crash slightly more controlled under weird circumstances... 2014-05-24 12:08:28 +02:00
Jakob Borg
1a231d39a5 Default permission bits are 0666 2014-05-24 08:53:54 +02:00
Jakob Borg
17e3d14272 Correct formatting of warning messages 2014-05-24 08:26:05 +02:00
Jakob Borg
03182c7714 Get tests in line with reality 2014-05-23 15:54:45 +02:00
Jakob Borg
963078f6ac Don't reuse certificate serials 2014-05-23 14:43:17 +02:00
Jakob Borg
8356b58b1d Implement IgnorePerms 2014-05-23 14:31:16 +02:00
Jakob Borg
303ce02271 Internal support for ignoring permissions 2014-05-23 13:10:26 +02:00
Jakob Borg
bcdc3ecdae There should be only One 2014-05-23 12:55:24 +02:00
Jakob Borg
b60d648e22 Convenience functions for flag testing 2014-05-23 12:53:26 +02:00
Jakob Borg
7bc36cbbd1 Add bit 17, No Permission Bits 2014-05-23 12:53:11 +02:00
Jakob Borg
04130fcb15 Allow GUI development with standard binary 2014-05-22 16:12:19 +02:00
Jakob Borg
52d8e4c691 Set local discovery port in GUI 2014-05-22 09:38:11 +02:00
Jakob Borg
ae0193b724 Configurable local announcement port (fixes #256) 2014-05-22 09:35:54 +02:00
Jakob Borg
2e1c33206f Fix discosrv build, build as part of all (fixes #257) 2014-05-22 08:46:19 +02:00
Jakob Borg
0c642ec7cf Un-ignore Godeps 2014-05-21 22:23:18 +02:00
Jakob Borg
b3ca96eeba Merge pull request #255 from KayoticSully/master
Resolves Issue #239
2014-05-21 22:21:33 +02:00
Jakob Borg
ae0e033178 Add KayoticSully 2014-05-21 22:20:53 +02:00
Ryan Sullivan
a97985b428 Added suggestions to settings fix. 2014-05-21 15:54:16 -04:00
Ryan Sullivan
63c0f11458 Merge remote-tracking branch 'upstream/master'
Conflicts:
	auto/gui.files.go
2014-05-21 15:15:37 -04:00
Jakob Borg
5b0dd9d3b2 Assets update 2014-05-21 21:09:31 +02:00
Jakob Borg
8bba82a08d Squelch interpolation errors at startup 2014-05-21 21:07:37 +02:00
Jakob Borg
51bf15728a Fix delete node (fixes #250) 2014-05-21 21:06:20 +02:00
Ryan Sullivan
b336b2c336 Merge remote-tracking branch 'upstream/master'
Conflicts:
	auto/gui.files.go
2014-05-21 14:38:54 -04:00
Jakob Borg
2331089854 Correct list of debugging facilities 2014-05-21 20:36:21 +02:00
Ryan Sullivan
8a5a573851 Fixed issue #239 Saving an unchanged config does not prompt for reboot 2014-05-21 14:35:51 -04:00
Jakob Borg
6fb05fc82a Add Edit > Show ID with QR (fixes #243) 2014-05-21 20:06:14 +02:00
Ryan Sullivan
358862c7ad Ignore sublime files and Godeps changes 2014-05-21 13:50:06 -04:00
Jakob Borg
4235175966 Show 'shutdown complete' modal (fixes #251) 2014-05-21 19:35:56 +02:00
Jakob Borg
cd433a4f52 Remove unneccessary verbage 2014-05-21 17:59:59 +02:00
Jakob Borg
f3b550744f Tweak GUI icons 2014-05-21 15:37:28 +02:00
Jakob Borg
f31e5eaba2 Capture integration test output 2014-05-21 15:23:54 +02:00
Jakob Borg
7c8652b600 Implement HTTPS for GUI 2014-05-21 14:42:53 +02:00
Jakob Borg
30837a7d95 Tweak icons and spacing 2014-05-20 21:34:30 +02:00
Jakob Borg
3c25a74f3b Correct units for rate limits (ref #234) 2014-05-20 21:16:04 +02:00
Jakob Borg
da3177dbe0 Removing a nonexistent file is success 2014-05-20 20:27:21 +02:00
Jakob Borg
3ca8dce98b Don't load files from index into suppressed state 2014-05-20 20:26:44 +02:00
Jakob Borg
e17fa7b14e Don't attempt to fetch suppressed files 2014-05-20 20:26:14 +02:00
Jakob Borg
230f149b4b Explain tilde in repo editor 2014-05-20 19:44:40 +02:00
Jakob Borg
dd3d8a6c98 Revamp UI (fixes #185, fixes #147, fixes #136, fixes #124) 2014-05-20 19:36:37 +02:00
Jakob Borg
36bdd9cb4d Add bootstrap-3.1.1 2014-05-20 18:41:31 +02:00
Jakob Borg
95f5e5fa9a Don't deadlock while sending and receiving large indexes 2014-05-20 18:41:01 +02:00
Jakob Borg
1aef03288a Revert "Show liveness indicator (beating heart)"
This reverts commit 84c4298cd5.

Conflicts:
	gui/app.js
	gui/index.html
2014-05-20 01:02:24 +02:00
Jakob Borg
a8c510cc72 Find correct upgrade asset for arm (fixes #232) 2014-05-20 00:06:16 +02:00
Jakob Borg
e359b146aa Properly handle read-write/read-only windows<->posix (fixes #236) 2014-05-19 23:42:08 +02:00
Jakob Borg
dba40eefb1 Fix handling of changed/deleted directories (fixes #231) 2014-05-19 22:31:28 +02:00
Jakob Borg
23b55f68b7 Also sync changes to only permission bits (fixes #237) 2014-05-19 16:24:39 +02:00
Jakob Borg
5c65e10875 Also remove dashes from GUI entered node IDs 2014-05-18 22:40:25 +02:00
Jakob Borg
d7df11e724 Failing integration test for folders + readonly (ref #231) 2014-05-18 14:07:22 +02:00
Jakob Borg
c08ce5c571 Test script for folder create/delete 2014-05-18 13:20:45 +02:00
Jakob Borg
12a6682eaf Account for 'size' of directories and deleted files when calculating in sync percentages 2014-05-18 12:58:36 +02:00
Jakob Borg
bbefcef53b Allow node IDs with spaces/dashes in config. Use dashes in GUI. (ref #230) 2014-05-18 12:05:31 +02:00
Jakob Borg
729b0143e1 Update assets for previous commit 2014-05-17 13:58:58 +02:00
Jakob Borg
84c4298cd5 Show liveness indicator (beating heart) 2014-05-17 13:56:59 +02:00
Jakob Borg
7d672fd989 Return an empty error list instead of null after clear 2014-05-17 13:55:10 +02:00
Jakob Borg
1442d6f4c2 Slightly less verbose xdr trace 2014-05-17 13:55:10 +02:00
Jakob Borg
13022817d4 Add xdr level tracing 2014-05-17 08:43:01 +02:00
Jakob Borg
e788a99c3e Don't fail build on 'go vet' errors (ref #227) 2014-05-16 20:05:44 +02:00
Jakob Borg
09d4b3f71e Log connection errors at INFO level (fixes #225) 2014-05-16 19:10:16 +02:00
Jakob Borg
78f0f2b131 Don't crash on error response to UPnP query (fixes #226) 2014-05-16 19:04:32 +02:00
Jakob Borg
40c750141a Actually announce listen port locally 2014-05-16 16:28:52 +02:00
Jakob Borg
b60251b960 Show node name in title/header (fixes #221) 2014-05-16 14:24:32 +02:00
Jakob Borg
958c39ef5f Don't spam console with debug 2014-05-16 14:24:31 +02:00
Jakob Borg
e22ddae3a8 Repair test suite 2014-05-15 09:09:21 -03:00
Jakob Borg
68afc897d6 Quote default flag parameters 2014-05-15 00:42:40 -03:00
Jakob Borg
ba58e95f6b Package level comments 2014-05-15 00:40:17 -03:00
Jakob Borg
adbd0b1834 Rename mc -> beacon 2014-05-15 00:33:40 -03:00
Jakob Borg
3e34fc66e6 Refactor model into separate package 2014-05-15 00:33:40 -03:00
Jakob Borg
f8e34c083e Refactor config into separate package 2014-05-14 21:18:09 -03:00
Jakob Borg
cba554d0fa Refactor logging into separate package 2014-05-14 21:08:56 -03:00
Jakob Borg
8903825e02 Use UDP broadcasts instead of multicast for discovery 2014-05-14 15:26:10 -03:00
Jakob Borg
81cd84add2 Merge pull request #216 from jedie/master
Change default ListenAddress to "0.0.0.0:22000"
2014-05-14 17:49:41 +02:00
Jens Diemer
8229d47da5 Update config_test.go 2014-05-14 17:43:49 +02:00
Jens Diemer
8f14d11d66 change default ListenAddress to "0.0.0.0:22000" 2014-05-14 17:43:23 +02:00
Jakob Borg
76f82cbd1f Make duplicate ID:s temporarily unique (fixes #153) 2014-05-14 07:58:33 -03:00
Jakob Borg
dc2f83e522 Add GUI validations for node and repo editors (fixes #153) 2014-05-14 07:55:00 -03:00
Jakob Borg
8b4282fe28 Merge pull request #213 from jedie/master
Documentation link in WebGUI is wrong.
2014-05-14 14:53:18 +02:00
Jakob Borg
c6416b235b Add jedie 2014-05-14 09:53:10 -03:00
Jens Diemer
79f2b3bd7e Update Documentation link. 2014-05-14 09:32:11 +02:00
Jakob Borg
1f996afa21 Merge pull request #202 from veeti/node-id-readability
Split node ID's into multiple parts/chunks for readability
2014-05-14 01:53:20 +02:00
Jakob Borg
21335d65c4 Do initial repository scan in parallel (ref #210) 2014-05-13 20:42:12 -03:00
Veeti Paananen
870ce57005 Split node ID's into multiple parts/chunks for readability
Helps with manual entry.
2014-05-13 23:43:14 +03:00
Jakob Borg
0d3caa2183 Increase file limit from 100.000 to 1.000.000 2014-05-13 10:05:36 -03:00
Jakob Borg
602d7e8d18 Merge pull request #207 from jpjp/patch-2
Update README.md
2014-05-13 14:54:15 +02:00
Jakob Borg
a2309f3119 Merge pull request #206 from jpjp/patch-1
Update PROTOCOL.md
2014-05-13 14:52:21 +02:00
jpjp
9a51be8548 Update README.md
Correct a couple of typos.
2014-05-13 14:46:58 +02:00
jpjp
5ed319ea42 Update PROTOCOL.md
Correct typos
2014-05-13 14:45:32 +02:00
Jakob Borg
eb7a70a3c9 Merge pull request #203 from jaseg/patch-1
PROTOCOL.md: Fixed typo
2014-05-13 12:17:01 +02:00
jaseg
b61f418bf2 PROTOCOL.md: Fixed typo 2014-05-13 07:11:09 +02:00
Jakob Borg
1338b0a2f8 Merge pull request #200 from veeti/fix-no-repositories
Return a blank array instead of null if there are no repositories
2014-05-13 05:37:26 +02:00
Jakob Borg
587e1a4f4c Add veeti 2014-05-13 00:36:30 -03:00
Veeti Paananen
85d5449b3c Return a blank array instead of null if there are no repositories
Fixes a bug where it's impossible to add repositories in the web
interface if none are defined.
2014-05-13 05:57:38 +03:00
Jakob Borg
532b576fd5 Expose discovery cache over rest interface 2014-05-12 22:08:55 -03:00
Jakob Borg
dd1197236d Provide discovery hint from the outside (ref #192) 2014-05-12 21:51:12 -03:00
Jakob Borg
e8a9abaf40 Empty directories are invalid (ref #188) 2014-05-12 21:30:04 -03:00
Jakob Borg
1bf07d6b58 Write response before shutting down 2014-05-12 21:15:18 -03:00
Jakob Borg
30ea9cb37e Use rest/shutdown to stop 2014-05-12 20:48:13 -03:00
Jakob Borg
bae9247d84 Add guidev build mode 2014-05-12 20:04:49 -03:00
Jakob Borg
a105ad1391 Easy godep/go vet setup. 2014-05-12 20:00:57 -03:00
Jakob Borg
abbb40abd2 Don't deadlock on closing while sending index (fixes #189) 2014-05-11 21:35:44 -03:00
Jakob Borg
20b23338f7 Shutdown from GUI (ref #192) 2014-05-11 20:16:27 -03:00
Jakob Borg
0fcbee6478 Don't serialize deprecated config options 2014-05-11 20:13:22 -03:00
Jakob Borg
1d602b9efa Enable discovery gossiping 2014-05-11 19:55:43 -03:00
Jakob Borg
b783169c72 Multicast test utility 2014-05-11 18:43:25 -03:00
Jakob Borg
e4f266883a better mc debug output 2014-05-11 18:20:14 -03:00
Jakob Borg
7a41362d90 Tagged date is that of the commit, not build 2014-05-11 17:26:48 -03:00
Jakob Borg
3ed783983f Don't build stcli by default 2014-05-11 15:40:14 -03:00
Jakob Borg
837f3a68ab Sort repos on directory (fixes #178) 2014-05-11 15:25:06 -03:00
Jakob Borg
59e45c5c68 Auto assign ports for GUI and BEP on initial startup (fixes #176) 2014-05-11 15:21:41 -03:00
Jakob Borg
94761d0472 Don't warn about legit requests for deleted files (fixes #173) 2014-05-11 14:54:26 -03:00
Jakob Borg
a91eb701bf Reload configuration after lost connection or restart. 2014-05-11 14:43:38 -03:00
Jakob Borg
1a1f118f1a Restructure protocol code with less locking 2014-05-11 14:30:29 -03:00
Jakob Borg
b115fca8a9 Increase ping timeout 2014-05-11 14:30:15 -03:00
Jakob Borg
dfd239ac06 Also build windows-386 and linux-armv5 2014-05-06 08:13:56 -03:00
Jakob Borg
2ae218d069 Rephrase Read Only into Repository Master (fixes #159) 2014-05-04 20:01:33 -03:00
Jakob Borg
1401d2ee9b Remove refereces to JS source mapping files 2014-05-04 18:27:09 +02:00
Jakob Borg
f39e105101 Stop repository if the directory disappears (fixes #154) 2014-05-04 18:22:59 +02:00
Jakob Borg
482795bab0 Streamline error handling and locking, with fix for close() race 2014-05-04 18:22:25 +02:00
Jakob Borg
10e8861f14 Smarter initial index sending 2014-05-04 18:21:57 +02:00
Jakob Borg
ecc6476308 Revert "Fix protocol close test"
This reverts commit 92c1ce57a6.
2014-05-04 08:16:45 +02:00
Jakob Borg
28e347002a Revert "Streamline error handling and locking" (fixes #172)
This reverts commit 116f232f5a.
2014-05-04 08:11:06 +02:00
Jakob Borg
b3d19bd5cc Run vet when building 2014-05-02 21:59:36 +02:00
Jakob Borg
647165ab89 Remove dead code 2014-05-02 21:59:18 +02:00
Jakob Borg
6807d9bd4c Fix upgrade non-support on Windows 2014-05-02 20:19:21 +02:00
Jakob Borg
699ecc7140 Some places should use RLock instead of Lock (ref #169) 2014-05-02 17:15:04 +02:00
Jakob Borg
b374ec9355 Save temporary in correct dir during upgrade 2014-05-02 17:04:45 +02:00
Jakob Borg
9659d021cb Merge pull request #170 from andrew-d/patch-1
Fix typo in header name
2014-05-02 16:59:08 +02:00
Jakob Borg
a4ad9eb134 Add andrew-d 2014-05-02 16:58:55 +02:00
Andrew
a455258a62 Fix typo in header name 2014-05-02 01:26:12 -07:00
Jakob Borg
0ae342673a Update saved dependencies 2014-05-02 10:05:48 +02:00
Jakob Borg
33d75a264d Built in upgrade functionality 2014-05-02 10:01:09 +02:00
Jakob Borg
89dc5bb951 Windows doesn't have SysProcAttr 2014-05-02 08:57:34 +02:00
Jakob Borg
45403917de Minor cleanup in discovery 2014-05-02 08:53:19 +02:00
Jakob Borg
ed476271a6 Start xdg-open in new process group (fixes #164) 2014-05-02 08:53:05 +02:00
Jakob Borg
1e92c47960 Don't bother starting without GUI (fixes #156) 2014-04-30 22:52:38 +02:00
Jakob Borg
4f2fe07ae4 Show node ID in regular text not disabled control (fixes #162) 2014-04-30 22:42:39 +02:00
Jakob Borg
aff3cd01c5 Don't show Offline badge when global disco is disabled (fixes #167) 2014-04-30 22:17:43 +02:00
Jakob Borg
ac74ee1468 Don't redirect to absolute URL (fixes #166) 2014-04-30 22:10:13 +02:00
Jakob Borg
0d55cf4be5 Don't use absolute URL for rest calls (fixes #166) 2014-04-30 22:02:34 +02:00
Jakob Borg
5399a25532 Getting started 2014-04-30 16:13:29 +02:00
Jakob Borg
ae882c93c9 Links to discourse 2014-04-30 15:14:42 +02:00
Jakob Borg
f398ca77c1 Better trace output from mc 2014-04-30 15:13:54 +02:00
Jakob Borg
dcd7d278aa Handle and indicate duplicate repo ID:s (fixes #153) 2014-04-27 21:53:27 +02:00
Jakob Borg
89f5f3bf9a Fix small data races 2014-04-27 21:33:57 +02:00
Jakob Borg
76ef42ee07 No drone.io badge 2014-04-27 13:37:53 +02:00
Jakob Borg
92c1ce57a6 Fix protocol close test 2014-04-27 13:25:35 +02:00
Jakob Borg
116f232f5a Streamline error handling and locking 2014-04-27 13:10:50 +02:00
Jakob Borg
ef81a36654 Extract method closeFile 2014-04-27 12:14:53 +02:00
Jakob Borg
9fd2724d73 Simplify requestSlots filling 2014-04-27 12:06:11 +02:00
Jakob Borg
07d49b61d0 Debug utility to print index file 2014-04-25 08:28:56 +02:00
Jakob Borg
0c4e6ae7de Safety: don't start if repo dir is missing (ref #154) 2014-04-24 10:27:43 +02:00
Jakob Borg
65ec129dfb Only create default config if it is actually missing (fixes #139) 2014-04-23 10:28:36 +02:00
Jakob Borg
3e4d628f54 Handle non-word characters in repo name (fixes #152) 2014-04-23 10:04:25 +02:00
Jakob Borg
71684bfa45 Use a more lenient cluster config check (fixes #148) 2014-04-22 16:42:25 +02:00
Jakob Borg
e73b7e0398 Show properly formatted time (fixes #149) 2014-04-22 15:59:16 +02:00
Jakob Borg
35ebdc76ff Hide temporary files on Windows (fixes #146) 2014-04-22 14:27:31 +02:00
Jakob Borg
90d0896848 Change default config directory (fixes #145) 2014-04-22 14:27:09 +02:00
Jakob Borg
5528db9693 Fix config test (hostname check) 2014-04-22 12:06:32 +02:00
Jakob Borg
aa78fbb09d Don't offer to delete this node (fixes #144) 2014-04-22 12:01:09 +02:00
Jakob Borg
d53b193e09 Ensure sensible node config on load (fixes #143) 2014-04-22 11:46:08 +02:00
Jakob Borg
e0e16c371f Don't include test utils in testing 2014-04-22 08:27:00 +02:00
Jakob Borg
53cd877899 More portable hostname 2014-04-22 08:25:40 +02:00
Jakob Borg
1207223f3d Report rates over the wire, not uncompressed 2014-04-21 12:49:47 +02:00
Jakob Borg
39be6932b5 discosrv: Better statistics 2014-04-19 23:14:56 +02:00
Jakob Borg
44a194d226 discosrv: Remove deprecated v1 support 2014-04-19 23:02:14 +02:00
Jakob Borg
9349eb77cd Let absence of password be absence 2014-04-19 22:36:24 +02:00
Jakob Borg
c64549471a Include build user and host in long version 2014-04-19 16:44:28 +02:00
Jakob Borg
264bcbc78c Always print long version at startup 2014-04-19 16:40:19 +02:00
Jakob Borg
f76fe1ac7a Include build date in -version output 2014-04-19 16:38:11 +02:00
Jakob Borg
6364c4ff3f Save bcrypt hash of password (fixes #138) 2014-04-19 13:33:51 +02:00
Jakob Borg
292a50de04 Use pseudo-random high port for UPnP mapping 2014-04-18 14:09:54 +02:00
Jakob Borg
a08cba9c85 Config option to enable/disable UPnP 2014-04-18 13:39:51 +02:00
Jakob Borg
9fb60d6935 UPnP Port Mapping (fixes #79) 2014-04-18 13:28:51 +02:00
Jakob Borg
f2ed2d98d8 Updated assets for previous commit 2014-04-17 10:56:33 +02:00
Jakob Borg
b802cb1e36 Show status of global announce server (fixes #71) 2014-04-16 17:36:09 +02:00
Jakob Borg
31bfd8c039 Decouple local from global announcing (fixes #132) 2014-04-16 16:49:01 +02:00
Jakob Borg
f72ee7a69e Set name of first node to the local hostname (fixes #121) 2014-04-16 16:35:29 +02:00
Jakob Borg
a98d75edaa Clear acknowledged errors server-side as well (fixes #128) 2014-04-16 16:30:49 +02:00
Jakob Borg
622568c327 Handle static addresses without port (fixes #131) 2014-04-16 15:28:45 +02:00
Jakob Borg
1ca7e47fd6 Show restarting notification instead of network error (fixes #129) 2014-04-16 15:16:44 +02:00
Jakob Borg
116203aef8 discosrv: Clean up debug logging 2014-04-16 15:06:54 +02:00
Jakob Borg
1bf128612d Prevent GUI from rendering before it's ready (fixes #127) 2014-04-15 19:14:46 +02:00
Jakob Borg
935a8eb9a7 Sort nodes on name if set, otherwise ID (fixes #119) 2014-04-15 10:57:17 +02:00
Jakob Borg
7e5b350096 Explanatory tooltips on data (fixes #118) 2014-04-15 10:34:34 +02:00
Jakob Borg
5c75869e85 Fixup integration test script 2014-04-14 15:33:09 +02:00
Jakob Borg
fbd5ddea72 Keep syncthing running during integration test 2014-04-14 13:45:47 +02:00
Jakob Borg
bc8e033eb5 Fix deadlock asshattery that snuck in somehow 2014-04-14 13:01:21 +02:00
Jakob Borg
70fa5ffa06 Add STCPUPROFILE env 2014-04-14 12:13:50 +02:00
Jakob Borg
fb162ff529 More up to date CPU usage indicator 2014-04-14 12:02:40 +02:00
Jakob Borg
9d535b13cf Speed up Global() at the price of Update() in fileset 2014-04-14 11:44:29 +02:00
Jakob Borg
48bfc2d9ed Show current repository state (fixes #89) 2014-04-14 09:58:17 +02:00
Jakob Borg
5064f846fc New Cluster Configuration message replaces Options (fixes #63) 2014-04-13 15:28:26 +02:00
Jakob Borg
41c228cb56 Faster and leaner comparisons in fileset 2014-04-13 14:30:55 +02:00
Jakob Borg
e974c8f33e Preallocate slices in fileset 2014-04-13 14:30:50 +02:00
Jakob Borg
94f5d5b59e Better fileset benchmarks 2014-04-13 12:41:54 +02:00
Jakob Borg
433a0cb9cc New GUI (fixes #73, fixes #87) 2014-04-09 23:00:23 +02:00
Jakob Borg
c42a6b511c Return repo-specific data in REST interface 2014-04-09 22:03:30 +02:00
Jakob Borg
346b6f4f11 Ensure deterministic test outcome 2014-04-09 10:24:43 +02:00
Jakob Borg
07eb4020bd Add github.com/codegangsta/martini-contrib/auth dep 2014-04-09 10:17:19 +02:00
Jakob Borg
c2f0c2225a Add tests for ReadBytesMaxInto(..., nil) 2014-04-09 10:09:41 +02:00
Jakob Borg
711587492c Add limit tests for ReadBytesMaxInto 2014-04-09 09:49:47 +02:00
Jakob Borg
536613f008 Fix config tests after deprecating options 2014-04-08 21:36:03 +02:00
Jakob Borg
c832fc7d1b Check entire node list for matches on connect 2014-04-08 21:32:58 +02:00
Jakob Borg
0b654581b6 Do not crash on TLS connection without certificate 2014-04-08 21:31:23 +02:00
Jakob Borg
cbae64fc06 GUI Basic Authentication (fixes #90) 2014-04-08 15:56:12 +02:00
Jakob Borg
1443a1388e Update dependencies 2014-04-08 15:16:46 +02:00
Jakob Borg
a203d99182 Build command for assets, use godep for tests etc 2014-04-08 15:16:13 +02:00
Jakob Borg
bc5ff6e1b6 Build command to update deps 2014-04-08 15:14:36 +02:00
Jakob Borg
3b3c0c5950 Configuration version 2 (nodes separate from repos) 2014-04-08 13:45:18 +02:00
Jakob Borg
ee0ee0e39d Remove deprected .ini format reader 2014-04-06 21:48:39 +02:00
Jakob Borg
8dee10ba9c Remove followSymlinks option (ref #92) 2014-04-06 21:44:17 +02:00
Jakob Borg
d3915b8dbf discosrv: Remove duplicate logging of limiter cache entries 2014-04-04 12:00:52 +02:00
Jakob Borg
671d5cace6 discosrv: Source based rate limiting 2014-04-03 23:40:10 +02:00
Jakob Borg
aa3d73d322 discosrv: Refactor handler loop 2014-04-03 23:40:03 +02:00
Jakob Borg
d30a286f38 Command line flag and REST command to reset and resync (fixes #85)
Still needs implemention in GUI
2014-04-03 22:10:51 +02:00
Jakob Borg
15699a39cf Synchronize directory existence & metadata (fixes #11) 2014-04-02 11:57:18 +02:00
Jakob Borg
a1f32095df Rate limit sent data, not uncompressed 2014-04-01 20:36:54 +02:00
Jakob Borg
76e0960a51 Streamline rate limiting 2014-04-01 14:22:38 +02:00
Jakob Borg
8e33288156 Also build 32 bit Linux (fixes #108) 2014-04-01 10:48:09 +02:00
Jakob Borg
f4d3a9980f Add link to discussion forum 2014-03-31 11:45:18 +02:00
Jakob Borg
fc8ce7c6e0 Include build env info in -version 2014-03-31 06:51:01 +02:00
Jakob Borg
aaf0604601 Remove pointless -v option 2014-03-31 06:47:08 +02:00
Jakob Borg
dddf563105 Build ARMv6 as well for RaspberryPi 2014-03-31 05:59:40 +02:00
Jakob Borg
dbe12cca4b Give the default repository an ID (fixes #106) 2014-03-30 23:11:55 +02:00
Jakob Borg
ddf0ddbd05 Repair model tests 2014-03-30 21:59:40 +02:00
Jakob Borg
5eb5a056bf Basic support for synchronizing multiple repositories (fixes #35) 2014-03-30 21:28:13 +02:00
Jakob Borg
2a5c0646c0 Format list of cipher suites 2014-03-29 18:41:06 +01:00
Jakob Borg
7d3c51df9e Spurious whitespace 2014-03-29 18:40:25 +01:00
Jakob Borg
a67bb5e720 Link to contributions needed 2014-03-29 18:36:16 +01:00
Jakob Borg
6d314cdc04 Spec clarifications and tightening 2014-03-29 18:34:09 +01:00
Jakob Borg
1139ea2c81 Expose a bit more information about needed file in REST interface 2014-03-29 14:58:44 +01:00
Jakob Borg
f87b1520e8 The Great Rewrite (fixes #36, #61, #94, #101)
Rewrite of the file model and pulling mechanism. Needs lots of cleanup
and bugfixes, now...
2014-03-29 13:47:21 +01:00
Jakob Borg
3700eb1e61 Use IPv4 multicast discovery 2014-03-28 11:04:48 +01:00
Jakob Borg
17a21102b3 Normalize asset paths (fixes #103) 2014-03-26 21:09:53 +01:00
Jakob Borg
e37441627f Merge pull request #104 from filoozom/patch-2
Set cache control on /rest/* (fixes #95)
2014-03-26 20:41:00 +01:00
Philippe Schommers
f4c6cd1676 Set cache control on /rest/* (fixes #95)
Disable caching for all /rest/* requests.
2014-03-26 20:32:35 +01:00
Jakob Borg
bcd3fd40e4 Add licensing and contribution info 2014-03-24 14:57:14 +01:00
Jakob Borg
d3d1a79996 Add branch and tag info 2014-03-24 14:55:13 +01:00
Jakob Borg
fb4a2c9b5a Don't use 'which' to check for godep 2014-03-24 07:37:26 +01:00
Jakob Borg
ff18a2c3e2 Harsher integration test 2014-03-23 09:08:48 +01:00
Jakob Borg
589244f39e Prevent zombie connections due to simultaneous connect 2014-03-23 08:53:34 +01:00
Jakob Borg
804cce7ba0 Ensure that we make progress on the read side of a connection 2014-03-23 08:44:27 +01:00
Jakob Borg
6d7b001b0b Test syncing large files 2014-03-23 08:43:18 +01:00
Jakob Borg
75cfa4c33e Update dependencies 2014-03-22 21:58:10 +01:00
Jakob Borg
145c8e4063 Only sign if the key is present 2014-03-22 21:38:01 +01:00
Jakob Borg
52cad94e86 Use godep 2014-03-22 21:33:18 +01:00
Jakob Borg
89399092b9 rm -r files 2014-03-22 21:25:59 +01:00
Jakob Borg
168b23556a Improve TestClose 2014-03-22 17:10:26 +01:00
Jakob Borg
513100bb92 Fix tests for >1 CPU (fixes #99) 2014-03-22 17:06:15 +01:00
Jakob Borg
68d9454bc4 Enforce correct filename normalization (fixes #96) 2014-03-21 09:09:01 +01:00
Jakob Borg
a4e56caf78 Include signature in release bundle (ref #97) 2014-03-21 07:40:48 +01:00
Jakob Borg
a84ea70387 Also remove old signatures when rebuilding 2014-03-20 17:56:43 +01:00
Jakob Borg
25acc0e445 Disable Windows builds for now (ref #94, #95) 2014-03-20 17:55:42 +01:00
Jakob Borg
16e1a4397a Sign builds, upload signatures with release artefacts (fixes #97) 2014-03-20 17:55:16 +01:00
Jakob Borg
8db1bf9732 Fix local announce (IPv6 multicast, include all listen addresses) 2014-03-19 13:57:19 +01:00
Jakob Borg
58fd379e35 Fast building assets 2014-03-17 22:57:19 +01:00
Jakob Borg
222272b75c Faster and more portable assets generator 2014-03-17 22:56:07 +01:00
Jakob Borg
ef8cf3bc30 Merge pull request #91 from qbit/master
Remove hard coded path for bash
2014-03-17 21:37:16 +01:00
Aaron Bieber
a48bf50608 get bash path from env 2014-03-17 11:16:26 -06:00
Aaron Bieber
60ae665c0f get bash path from env 2014-03-17 11:15:59 -06:00
Jakob Borg
2df78a9313 Reimplement quick startup scan 2014-03-16 08:14:55 +01:00
Jakob Borg
d2d32f26c7 Don't count deleted files in synced bytes (fixes #88) 2014-03-14 11:12:34 +01:00
Jakob Borg
473df1bd19 Fix integration test 2014-03-14 11:12:34 +01:00
Jakob Borg
880f417ae3 Add CPU usage reporting on Solaris 2014-03-12 10:15:00 +01:00
Jakob Borg
043fa7f489 Add environment variable to inhibit restart 2014-03-12 10:12:35 +01:00
Jakob Borg
9b0768a71b Temporarily remove broken integration tests 2014-03-12 06:33:18 +01:00
Jakob Borg
446b21c568 Package descriptions for godoc 2014-03-12 06:32:26 +01:00
Jakob Borg
b3c2ffc96a Test for previous commit and never ignore entire repo directory 2014-03-09 12:36:26 +01:00
Jakob Borg
b5f652a815 Do not descend into ignored directories (fixes #86) 2014-03-09 12:21:07 +01:00
Jakob Borg
9ec7de643e Refactor profiler startup / logging 2014-03-09 09:18:28 +01:00
Jakob Borg
2553ba0463 Discover & main tracing 2014-03-09 09:15:36 +01:00
Jakob Borg
52ee7d5724 Discovery tracing 2014-03-09 08:58:03 +01:00
Jakob Borg
d4ef6a6285 Document env vars, start profiler based on STPROFILER 2014-03-09 08:48:29 +01:00
Jakob Borg
56b7d3c28d Don't start browser on restart 2014-03-09 08:35:53 +01:00
Jakob Borg
ae94b726a7 Don't expose a -delay paramater 2014-03-09 08:35:38 +01:00
Jakob Borg
a88e4db1ee Option to not start browser (fixes #84) 2014-03-08 23:19:33 +01:00
Jakob Borg
0ebd4a6ba1 Fix relative path open bug 2014-03-08 23:18:50 +01:00
Jakob Borg
1448cfe66a Refactor out file scanner into separate package 2014-03-08 23:02:01 +01:00
Jakob Borg
d6c9afd07f Fix handling of default values in config (fixes #83) 2014-03-04 22:29:48 +01:00
Jakob Borg
799f55e7ae Add basic config tests 2014-03-04 22:17:39 +01:00
Jakob Borg
04a3db132f Merge pull request #81 from filoozom/patch-1
Fix isTempName to work on Windows (fixes #80)
2014-03-04 21:56:23 +01:00
Philippe Schommers
d06204959e Fix isTempName to work on Windows (fixes #80)
```path.Base()``` is for slash-separated paths, whereas Windows uses "\" to separate paths. Just convert the \ to / and it works.
2014-03-04 18:48:03 +01:00
Jakob Borg
2d0600de38 Merge pull request #78 from filoozom/patch-1
Update config.go to handle default slices correctly (fixes #76)
2014-03-04 15:09:47 +01:00
filoozom
6a1c055288 Delete comment in config.go 2014-03-04 15:02:26 +01:00
filoozom
b9ec30ebdb Update config.go 2014-03-04 11:29:51 +01:00
filoozom
428164f395 Update config.go to handle default slices correctly 2014-03-04 11:25:10 +01:00
Jakob Borg
ba59e0d3f0 Use undirected broadcast and WriteTo (fixes #75) 2014-03-03 18:19:32 +01:00
Jakob Borg
5d8f0f835e Merge branch 'filoozom-patch-1'
* filoozom-patch-1:
  Fix divided by zero when the sync folder is empty (tot = 0)
  Delete cfgFile before renaming it on Windows
  Set the right config and home dir for each OS
  Update getHomeDir() to use "os/user"
2014-03-03 14:06:15 +01:00
filoozom
b4a1aadd1b Fix divided by zero when the sync folder is empty (tot = 0) 2014-03-03 08:47:52 +01:00
filoozom
8f41d90ab1 Delete cfgFile before renaming it on Windows 2014-03-03 08:46:20 +01:00
Jakob Borg
9743386166 Re-add inadvertently ignored files 2014-03-02 23:58:24 +01:00
Jakob Borg
0afcb5b7e7 Clean up build.sh 2014-03-02 23:55:08 +01:00
filoozom
043dea760f Set the right config and home dir for each OS
Use %AppData%\syncthing for the config files and %USERPROFILE%\Sync as sync folder for Windows.
2014-03-02 23:49:51 +01:00
Jakob Borg
0618e2b9b4 Don't include timestamp in auto generated files 2014-03-02 23:15:56 +01:00
Jakob Borg
3c171d281c Move cmd files into subdir 2014-03-02 23:13:04 +01:00
Jakob Borg
c217b7cd22 Change default announce server to announce.syncthing.net 2014-03-02 17:29:35 +01:00
filoozom
23593c3d20 Update getHomeDir() to use "os/user"
os.Getenv("HOME") doesn't work properly on Windows (and maybe other systems?) and the package "os/user" gives us a convenient way to find the home directory for every OS.
2014-03-02 16:07:12 +01:00
Jakob Borg
192117dc11 Merge branch 'v0.6'
* v0.6:
  Open GUI on startup
2014-03-02 13:08:05 +01:00
Jakob Borg
24b8f9211a Open GUI on startup 2014-03-02 12:52:32 +01:00
Jakob Borg
51788d6f0e Add some support packages 2014-03-01 11:11:37 +01:00
Jakob Borg
ea0bed2238 drone.io badge 2014-02-24 14:06:22 +01:00
Jakob Borg
e2fe57c440 deadcode 2014-02-24 13:34:24 +01:00
Jakob Borg
434a0ccf2a golint 2014-02-24 13:29:30 +01:00
Jakob Borg
e7bf3ac108 go vet 2014-02-24 13:24:03 +01:00
Jakob Borg
c5bdaebf2b Remove spurious debug print 2014-02-23 15:08:15 +01:00
Jakob Borg
645233e7dc Respond to local announces of new nodes 2014-02-23 15:01:37 +01:00
Jakob Borg
c6e396e8fb Send announcements to all connected networks (fixes #56) 2014-02-23 14:48:56 +01:00
Jakob Borg
a57e2b358f Fix tests 2014-02-23 13:58:10 +01:00
Jakob Borg
d0863d495c Use simple script for asset embedding 2014-02-23 13:54:23 +01:00
Jakob Borg
5837277f8d Rework XDR encoding 2014-02-20 17:42:17 +01:00
Jakob Borg
87d473dc8f Remove obsolete go-flags package 2014-02-17 09:27:35 +01:00
Jakob Borg
9744629c4b discosrv: Expire nodes, reduce debug logging 2014-02-17 09:23:37 +01:00
Jakob Borg
8f0a015abf Find syncthing binary in $PATH when restarting (fixes #68) 2014-02-17 08:47:48 +01:00
Jakob Borg
f89fa6caed Factor out XDR en/decoding 2014-02-15 12:08:55 +01:00
Jakob Borg
21a7f3960a Add multi-repository support to protocol (ref #35) 2014-02-13 12:52:47 +01:00
Jakob Borg
9f63feef30 Add peer node sync status in GUI (fixes #46) 2014-02-13 12:51:55 +01:00
Jakob Borg
c171780c0d Reorder locking to avoid deadlock (fixes #64) 2014-02-13 12:51:51 +01:00
Jakob Borg
5daf6ecf70 Actually embed GUI changes from 91d5c4a... 2014-02-13 09:27:06 +01:00
Jakob Borg
6c8135126d Initialize logging earlier (fix panic in tests) 2014-02-13 08:59:27 +01:00
Jakob Borg
91d5c4a1ae Show warnings in GUI (fixes #66) 2014-02-12 23:18:41 +01:00
Jakob Borg
2cbe81f1c7 Restart from web gui (fixes #50) 2014-02-12 12:10:44 +01:00
Jakob Borg
a26ce61d92 (Re)Fix locking around deleteFile (fixes #64) 2014-02-11 16:04:55 +01:00
Jakob Borg
478300f6d8 Only show address when connected (fixes #58)
The configured address is visible in the config dialog.
2014-02-11 14:34:47 +01:00
Jakob Borg
3a5b816125 Allow setting a friendly name for the local node (fixes #65) 2014-02-10 20:54:57 +01:00
Jakob Borg
b6814241cc Fix locking around deleteFile (fixes #64) 2014-02-09 23:24:55 +01:00
Jakob Borg
fc6eabea28 Enforce identical member configuration among nodes (fixes #63) 2014-02-09 23:13:06 +01:00
Jakob Borg
14b3791b2b Don't panic for legitimate file errors (fixes #55) 2014-02-09 22:41:30 +01:00
Jakob Borg
e6b29988e5 Logo 2014-02-07 22:33:58 +01:00
Jakob Borg
3cb7b8f22b Allow multiple listenAddresses (fixes #52) 2014-02-05 23:17:17 +01:00
Jakob Borg
2297e29502 Give friendly names to nodes (fixes #54) 2014-02-05 22:49:26 +01:00
Jakob Borg
ea41acfff5 Clarify status badges and fix column widths (fixes #53) 2014-02-05 22:42:23 +01:00
Jakob Borg
1aefc50e35 Always show local node, and summarize traffic stats (fixes #43) 2014-02-05 21:30:04 +01:00
Jakob Borg
9bd4fa5008 Make immediate write error only slightly less cryptic (fixes #51) 2014-02-05 20:58:39 +01:00
Jakob Borg
89c2f61b30 Reduce default verbosity now that the GUI is there 2014-02-03 16:22:15 +01:00
Jakob Borg
a1d575894a Edit configuration in GUI; use XML configuration 2014-02-03 15:42:59 +01:00
Jakob Borg
71def3a970 Don't include resource fork crap in builds (fixes #48) 2014-02-01 20:23:02 +01:00
Jakob Borg
13854250b3 Always show self in cluster list (fixes #43) 2014-02-01 11:22:41 +01:00
Jakob Borg
e6078f9449 Streamline build script 2014-02-01 10:10:07 +01:00
Jakob Borg
5980952495 Actually load index cache again (fixes #45) 2014-01-29 22:02:38 +01:00
Jakob Borg
618c376e18 Synchronize zero sized files (fixes #44) 2014-01-29 21:52:27 +01:00
Jakob Borg
d31a126408 CONTRIBUTING.md 2014-01-28 19:10:39 +01:00
Jakob Borg
6d3f8a2c06 Parallell -> parallel (ref #13) 2014-01-26 16:48:20 +01:00
Jakob Borg
b1ba976122 Move auto generated source to a package 2014-01-26 15:02:06 +01:00
Jakob Borg
81d5d1d4a6 Rework config/flags (fixes #13) 2014-01-26 14:45:03 +01:00
Jakob Borg
ea5ef28c5a Performance: improve need computation 2014-01-23 22:20:15 +01:00
Jakob Borg
fc2ebc6cad Performance: make filequeue not suck 2014-01-23 16:39:12 +01:00
Jakob Borg
01096fff6c Add version info to GUI (fixes #41) 2014-01-23 13:13:15 +01:00
Jakob Borg
2ea3558283 Add Options message to protocol 2014-01-23 13:12:45 +01:00
Jakob Borg
20a47695fb Create syncthing.ini template (fixes #39) 2014-01-22 14:28:14 +01:00
Jakob Borg
1dde9ec2d8 New file change suppression algorithm (fixes #30) 2014-01-22 12:52:27 +01:00
Jakob Borg
0841a46055 Don't crash on invalid options 2014-01-22 12:52:15 +01:00
Jakob Borg
84c0749d20 Slightly more compact GUI resources 2014-01-20 23:17:57 +01:00
Jakob Borg
6b02f9e44f Fix GUI files modtime (ish...) 2014-01-20 23:08:29 +01:00
Jakob Borg
84d7452f9e Use embed instead of nrsc, enables 'go get' 2014-01-20 23:01:38 +01:00
Jakob Borg
9b449cb527 Fix windows build (fixes #38) 2014-01-20 23:00:49 +01:00
Jakob Borg
d9ffd359e2 Tweak locking and integration test. 2014-01-20 22:22:27 +01:00
Jakob Borg
b67443eb40 Integration test 2014-01-20 07:38:57 +01:00
Jakob Borg
4ac204b604 Fine grained locking 2014-01-20 07:38:48 +01:00
Jakob Borg
fff50b5472 Delete deadlock 2014-01-14 17:47:27 -07:00
Jakob Borg
8d5aed410f Clear availability for disconnected node 2014-01-13 11:22:57 -07:00
Jakob Borg
ba0e4ded65 Deadlock fix and cleanups 2014-01-13 10:29:23 -07:00
Jakob Borg
f0b18685a5 Show 'this node' in GUI 2014-01-12 15:19:03 -07:00
Jakob Borg
fc2b557ae6 Don't print help twice 2014-01-12 14:47:04 -07:00
Jakob Borg
af399ae9f3 Cleanup ignore tests 2014-01-12 11:10:15 -07:00
Jakob Borg
45fcf4bc84 Implement new puller routine (fixes #33) 2014-01-12 11:02:16 -07:00
Jakob Borg
55f61ccb5e Simple send rate limit 2014-01-12 10:19:22 -07:00
Jakob Borg
b601fc5627 Don't build with CPU usage on Solaris 2014-01-10 15:32:30 +01:00
Jakob Borg
832c0ffad0 Report CPU/mem usage in GUI 2014-01-10 00:12:32 +01:00
Jakob Borg
cb33f27f23 Woops: reignore .stignore 2014-01-09 23:00:42 +01:00
Jakob Borg
92dee7c082 Only fetch deps, don't build 2014-01-09 23:00:23 +01:00
Jakob Borg
b9af45bc6b Prepopulate ignore patterns (fixes #21) 2014-01-09 22:46:01 +01:00
Jakob Borg
a18f6c6d90 Do go get as part of build unless fast build requested (fixes #31) 2014-01-09 21:22:05 +01:00
Jakob Borg
6e11e3cda9 Build for Linux on ARM (fixes #32) 2014-01-09 21:17:41 +01:00
Jakob Borg
2935aebe53 Benchmarking 2014-01-09 14:11:55 +01:00
Jakob Borg
71f78f0d62 Future proofing: handle file records with unknown flags 2014-01-09 11:04:42 +01:00
Jakob Borg
3e1194e5ff Show web GUI address on startup (fixes #27) 2014-01-09 10:40:12 +01:00
Jakob Borg
6d64992e64 Display alert on GUI connection error (fixes #26) 2014-01-09 10:31:27 +01:00
Jakob Borg
211180108e Tweak TLS settings (ref #23) 2014-01-09 09:30:22 +01:00
Jakob Borg
17e78d6f7e Option to show version (fixes #24) 2014-01-08 14:37:33 +01:00
Jakob Borg
1ef86379fb Actually send index updates for version bumps 2014-01-08 14:21:47 +01:00
Jakob Borg
884a7d6a1b Default to running GUI on 127.0.0.1:8080 2014-01-08 13:56:29 +01:00
Jakob Borg
334961fe10 Footer with links 2014-01-08 13:52:17 +01:00
Jakob Borg
2cfb24892f Add version and invalid bit to protocol 2014-01-07 22:44:21 +01:00
Jakob Borg
d4fe1400d2 Longer RSA key and stronger node ID hash (ref #23) 2014-01-07 22:04:30 +01:00
Jakob Borg
69ef4d261d Unbreak build script 2014-01-07 17:07:46 +01:00
Jakob Borg
91c102e4fe Syncronize file mode (fixes #20) 2014-01-07 16:38:07 +01:00
Jakob Borg
b4db177045 Allow deletes per default (fixes #19) 2014-01-07 16:15:18 +01:00
Jakob Borg
340c9095dd Suppress frequent changes to files (fixes #12) 2014-01-07 16:10:38 +01:00
Jakob Borg
e3bc33dc88 Move binary to build destination 2014-01-07 12:14:50 +01:00
Jakob Borg
eebc145055 Point to the wiki for documentation (fixes #10) 2014-01-07 12:07:56 +01:00
Jakob Borg
92b01fa48a Build tar file for current OS/architecture 2014-01-07 11:52:42 +01:00
Jakob Borg
2a0d1ab294 Merge pull request #18 from jpjp/patch-1
synch -> sync
2014-01-07 02:33:55 -08:00
jpjp
2bdab426ff synch -> sync 2014-01-06 22:27:57 +01:00
Jakob Borg
e769de9986 Announce read/write/delete status on startup (fixes #14) 2014-01-06 21:41:29 +01:00
Jakob Borg
4b11e66914 Verify requests against model (fixes #15) 2014-01-06 21:31:36 +01:00
Jakob Borg
28d3936a3c Add .stignore to testdata 2014-01-06 21:31:20 +01:00
Jakob Borg
986b15573a Ignore files matching patterns in .stignore (fixes #7) 2014-01-06 21:17:18 +01:00
Jakob Borg
46d828e349 Expand tilde in repository path (fixes #8) 2014-01-06 19:37:26 +01:00
Jakob Borg
48603a1619 Do quick build even on test failure 2014-01-06 15:50:15 +01:00
Jakob Borg
17d5f2bbfc Fix model test 2014-01-06 15:49:51 +01:00
Jakob Borg
b64af73607 Dead code removal 2014-01-06 12:45:43 +01:00
Jakob Borg
c9cce9613e Refactor out the model into a subpackage 2014-01-06 11:11:18 +01:00
Jakob Borg
1392905d63 Link to issues 2014-01-06 08:19:33 +01:00
Jakob Borg
271d7eedc4 Better progress bar calculation 2014-01-06 06:38:01 +01:00
Jakob Borg
ab8482a424 Add --trace-need 2014-01-06 06:12:40 +01:00
Jakob Borg
c8a14d1c3d Refactor how --delete affects things 2014-01-06 06:12:33 +01:00
Jakob Borg
8974c33f2f Move build artefacts dir 2014-01-06 06:11:19 +01:00
Jakob Borg
ed675a61d7 Prettify need table 2014-01-06 06:10:53 +01:00
Jakob Borg
60b00af0bb Workaround for bug in Solaris compiler 2014-01-06 05:57:41 +01:00
Jakob Borg
0ceddc4fa3 Redirect / to index.html 2014-01-06 00:05:07 +01:00
Jakob Borg
8c1996f7e5 Rudimentary HTTP GUI 2014-01-05 23:54:57 +01:00
Jakob Borg
6679c84cfb Refactor statistics printing 2014-01-05 16:20:18 +01:00
Jakob Borg
7b6f43cbb5 Merge pull request #6 from philips/patch-2
README: fix link to discover.go
2014-01-04 14:54:16 -08:00
Brandon Philips
c124989163 README: fix link to discover.go
Small markdown syntax error fix.
2014-01-03 09:43:42 -08:00
Jakob Borg
c549e413a2 Close tmpfiles earlier (ref #2) 2014-01-01 16:31:52 -05:00
Jakob Borg
63a05ff6fa Command line option to ignore index cache 2014-01-01 16:31:35 -05:00
Jakob Borg
89a5aac6ea Use gzip compression for index cache 2014-01-01 16:31:04 -05:00
Jakob Borg
232d715c37 Fix broken --cfg flag 2014-01-01 08:49:55 -05:00
Jakob Borg
1c4e710adc Build windows binaries 2014-01-01 08:18:11 -05:00
Jakob Borg
7fdea0dd93 Close even if we don't have a connection 2014-01-01 08:09:17 -05:00
Jakob Borg
5b84b72d15 Await completion of pull round before starting next (ref #2) 2014-01-01 08:02:12 -05:00
Jakob Borg
7e0be89052 Simplify index sending, prevent ping timeout 2013-12-31 21:22:49 -05:00
Jakob Borg
632bcae856 Mostly lock free receive loop 2013-12-30 22:10:54 -05:00
Jakob Borg
fd56123acf Send index in chunks of 1000 to avoid lengthy blocking 2013-12-30 22:05:00 -05:00
Jakob Borg
a2a2e1d466 Atomically replace local index cache 2013-12-30 22:04:30 -05:00
Jakob Borg
d4c5786a14 Change default queue parameters to optimize better for small files 2013-12-30 21:35:20 -05:00
Jakob Borg
42ad9f8b02 Increase ping timeout 2013-12-30 21:32:20 -05:00
Jakob Borg
0f6b34160c Propagate and log reason for connection close 2013-12-30 21:25:45 -05:00
Jakob Borg
7e3b29e3e0 Remove source info in log by default 2013-12-30 21:25:34 -05:00
Jakob Borg
2f660aff7a Improve no such node error messages 2013-12-30 20:55:33 -05:00
Jakob Borg
af3e64a5a7 Remove broken Ping latency measurement 2013-12-30 20:52:36 -05:00
Jakob Borg
9560265adc Always continue walk in the face of errors (fixes #1) 2013-12-30 19:50:04 -05:00
Jakob Borg
4097528aa2 Don't crash on zero nodes in pull 2013-12-30 19:49:25 -05:00
Jakob Borg
71d50a50f4 Make sure to always close directory fd 2013-12-30 19:30:59 -05:00
Jakob Borg
ec0489a8ea Improve log message consistency 2013-12-30 15:31:41 -05:00
Jakob Borg
7948d046d1 Fix locking around close events 2013-12-30 15:27:20 -05:00
Jakob Borg
223bdbb9aa Improve/fix buffer handling 2013-12-30 15:06:44 -05:00
507 changed files with 78826 additions and 8225 deletions

9
.gitignore vendored
View File

@@ -1,3 +1,10 @@
syncthing
syncthing.exe
stcli
stcli.exe
*.tar.gz
build
*.zip
*.asc
*.sublime*
discosrv
stpidx

55
CONTRIBUTING.md Normal file
View File

@@ -0,0 +1,55 @@
Please do contribute! If you want to contribute but are unsure where to
start, the [Contributions Needed
topic](http://discourse.syncthing.net/t/contributions-needed/49)
lists areas in need of attention.
## Licensing
All contributions are made under the same MIT License as the rest of the
project, except documentation which is licensed under the Creative
Commons Attribution 4.0 International License. You retain the copyright
to code you have written.
When accepting your first contribution, the maintainer of the project
will ensure that you are added to the CONTRIBUTORS file.
## Building
[See the
documentation](http://discourse.syncthing.net/t/building-syncthing/44)
## Branches
- `master` is the main branch containing good code that will end up in
the next release. You should base your work on it. It won't ever be
rebased or force-pushed to.
- `vx.y` branches exist to make patch releases on otherwise obsolete
minor releases. Should only contain fixes cherry picked from master.
Don't base any work on them.
- Other branches are probably topic branches and may be subject to
rebasing. Don't base any work on them unless you specifically know
otherwise.
## Tags
All releases are tagged semver style as `vx.y.z`. Release tags are
signed by GPG key BCE524C7.
## Tests
Yes please!
## Style
`go fmt`
## Documentation
[Over here!](http://discourse.syncthing.net/category/documentation)
## License
MIT

8
CONTRIBUTORS Normal file
View File

@@ -0,0 +1,8 @@
Aaron Bieber <qbit@deftly.net>
Andrew Dunham <andrew@du.nham.ca>
Brandon Philips <brandon@ifup.org>
James Patterson <jamespatterson@operamail.com>
Jens Diemer <github.com@jensdiemer.de>
Philippe Schommers <philippe@schommers.be>
Ryan Sullivan <kayoticsully@gmail.com>
Veeti Paananen <veeti.paananen@rojekti.fi>

65
Godeps/Godeps.json generated Normal file
View File

@@ -0,0 +1,65 @@
{
"ImportPath": "github.com/calmh/syncthing",
"GoVersion": "go1.2.2",
"Packages": [
"./cmd/syncthing",
"./cmd/assets",
"./discover/cmd/discosrv"
],
"Deps": [
{
"ImportPath": "bitbucket.org/kardianos/osext",
"Comment": "null-9",
"Rev": "364fb577de68fb646c4cb39cc0e09c887ee16376"
},
{
"ImportPath": "code.google.com/p/go.crypto/bcrypt",
"Comment": "null-185",
"Rev": "6478cc9340cbbe6c04511280c5007722269108e9"
},
{
"ImportPath": "code.google.com/p/go.crypto/blowfish",
"Comment": "null-185",
"Rev": "6478cc9340cbbe6c04511280c5007722269108e9"
},
{
"ImportPath": "code.google.com/p/go.text/transform",
"Comment": "null-81",
"Rev": "9cbe983aed9b0dfc73954433fead5e00866342ac"
},
{
"ImportPath": "code.google.com/p/go.text/unicode/norm",
"Comment": "null-81",
"Rev": "9cbe983aed9b0dfc73954433fead5e00866342ac"
},
{
"ImportPath": "github.com/codegangsta/inject",
"Rev": "9aea7a2fa5b79ef7fc00f63a575e72df33b4e886"
},
{
"ImportPath": "github.com/codegangsta/martini",
"Comment": "v0.1-142-g8659df7",
"Rev": "8659df7a51aebe6c6120268cd5a8b4c34fa8441a"
},
{
"ImportPath": "github.com/golang/groupcache/lru",
"Rev": "d781998583680cda80cf61e0b37dd0cd8da2eb52"
},
{
"ImportPath": "github.com/juju/ratelimit",
"Rev": "cbaa435c80a9716e086f25d409344b26c4039358"
},
{
"ImportPath": "github.com/vitrun/qart/coding",
"Rev": "ccb109cf25f0cd24474da73b9fee4e7a3e8a8ce0"
},
{
"ImportPath": "github.com/vitrun/qart/gf256",
"Rev": "ccb109cf25f0cd24474da73b9fee4e7a3e8a8ce0"
},
{
"ImportPath": "github.com/vitrun/qart/qr",
"Rev": "ccb109cf25f0cd24474da73b9fee4e7a3e8a8ce0"
}
]
}

5
Godeps/Readme generated Normal file
View File

@@ -0,0 +1,5 @@
This directory tree is generated automatically by godep.
Please do not edit.
See https://github.com/tools/godep for more information.

2
Godeps/_workspace/.gitignore generated vendored Normal file
View File

@@ -0,0 +1,2 @@
/pkg
/bin

View File

@@ -0,0 +1,20 @@
Copyright (c) 2012 Daniel Theophanes
This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:
1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source
distribution.

View File

@@ -0,0 +1,32 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Extensions to the standard "os" package.
package osext
import "path/filepath"
// Executable returns an absolute path that can be used to
// re-invoke the current program.
// It may not be valid after the current program exits.
func Executable() (string, error) {
p, err := executable()
return filepath.Clean(p), err
}
// Returns same path as Executable, returns just the folder
// path. Excludes the executable name.
func ExecutableFolder() (string, error) {
p, err := Executable()
if err != nil {
return "", err
}
folder, _ := filepath.Split(p)
return folder, nil
}
// Depricated. Same as Executable().
func GetExePath() (exePath string, err error) {
return Executable()
}

View File

@@ -0,0 +1,16 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package osext
import "syscall"
func executable() (string, error) {
f, err := Open("/proc/" + itoa(Getpid()) + "/text")
if err != nil {
return "", err
}
defer f.Close()
return syscall.Fd2path(int(f.Fd()))
}

View File

@@ -0,0 +1,25 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build linux netbsd openbsd
package osext
import (
"errors"
"os"
"runtime"
)
func executable() (string, error) {
switch runtime.GOOS {
case "linux":
return os.Readlink("/proc/self/exe")
case "netbsd":
return os.Readlink("/proc/curproc/exe")
case "openbsd":
return os.Readlink("/proc/curproc/file")
}
return "", errors.New("ExecPath not implemented for " + runtime.GOOS)
}

View File

@@ -0,0 +1,82 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin freebsd
package osext
import (
"os"
"path/filepath"
"runtime"
"syscall"
"unsafe"
)
var startUpcwd, getwdError = os.Getwd()
func executable() (string, error) {
var mib [4]int32
switch runtime.GOOS {
case "freebsd":
mib = [4]int32{1 /* CTL_KERN */, 14 /* KERN_PROC */, 12 /* KERN_PROC_PATHNAME */, -1}
case "darwin":
mib = [4]int32{1 /* CTL_KERN */, 38 /* KERN_PROCARGS */, int32(os.Getpid()), -1}
}
n := uintptr(0)
// get length
_, _, err := syscall.Syscall6(syscall.SYS___SYSCTL, uintptr(unsafe.Pointer(&mib[0])), 4, 0, uintptr(unsafe.Pointer(&n)), 0, 0)
if err != 0 {
return "", err
}
if n == 0 { // shouldn't happen
return "", nil
}
buf := make([]byte, n)
_, _, err = syscall.Syscall6(syscall.SYS___SYSCTL, uintptr(unsafe.Pointer(&mib[0])), 4, uintptr(unsafe.Pointer(&buf[0])), uintptr(unsafe.Pointer(&n)), 0, 0)
if err != 0 {
return "", err
}
if n == 0 { // shouldn't happen
return "", nil
}
for i, v := range buf {
if v == 0 {
buf = buf[:i]
break
}
}
var strpath string
if buf[0] != '/' {
var e error
if strpath, e = getAbs(buf); e != nil {
return strpath, e
}
} else {
strpath = string(buf)
}
// darwin KERN_PROCARGS may return the path to a symlink rather than the
// actual executable
if runtime.GOOS == "darwin" {
if strpath, err := filepath.EvalSymlinks(strpath); err != nil {
return strpath, err
}
}
return strpath, nil
}
func getAbs(buf []byte) (string, error) {
if getwdError != nil {
return string(buf), getwdError
} else {
if buf[0] == '.' {
buf = buf[1:]
}
if startUpcwd[len(startUpcwd)-1] != '/' && buf[0] != '/' {
return startUpcwd + "/" + string(buf), nil
}
return startUpcwd + string(buf), nil
}
}

View File

@@ -0,0 +1,79 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build darwin linux freebsd netbsd windows
package osext
import (
"fmt"
"os"
oexec "os/exec"
"path/filepath"
"runtime"
"testing"
)
const execPath_EnvVar = "OSTEST_OUTPUT_EXECPATH"
func TestExecPath(t *testing.T) {
ep, err := Executable()
if err != nil {
t.Fatalf("ExecPath failed: %v", err)
}
// we want fn to be of the form "dir/prog"
dir := filepath.Dir(filepath.Dir(ep))
fn, err := filepath.Rel(dir, ep)
if err != nil {
t.Fatalf("filepath.Rel: %v", err)
}
cmd := &oexec.Cmd{}
// make child start with a relative program path
cmd.Dir = dir
cmd.Path = fn
// forge argv[0] for child, so that we can verify we could correctly
// get real path of the executable without influenced by argv[0].
cmd.Args = []string{"-", "-test.run=XXXX"}
cmd.Env = []string{fmt.Sprintf("%s=1", execPath_EnvVar)}
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("exec(self) failed: %v", err)
}
outs := string(out)
if !filepath.IsAbs(outs) {
t.Fatalf("Child returned %q, want an absolute path", out)
}
if !sameFile(outs, ep) {
t.Fatalf("Child returned %q, not the same file as %q", out, ep)
}
}
func sameFile(fn1, fn2 string) bool {
fi1, err := os.Stat(fn1)
if err != nil {
return false
}
fi2, err := os.Stat(fn2)
if err != nil {
return false
}
return os.SameFile(fi1, fi2)
}
func init() {
if e := os.Getenv(execPath_EnvVar); e != "" {
// first chdir to another path
dir := "/"
if runtime.GOOS == "windows" {
dir = filepath.VolumeName(".")
}
os.Chdir(dir)
if ep, err := Executable(); err != nil {
fmt.Fprint(os.Stderr, "ERROR: ", err)
} else {
fmt.Fprint(os.Stderr, ep)
}
os.Exit(0)
}
}

View File

@@ -0,0 +1,34 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package osext
import (
"syscall"
"unicode/utf16"
"unsafe"
)
var (
kernel = syscall.MustLoadDLL("kernel32.dll")
getModuleFileNameProc = kernel.MustFindProc("GetModuleFileNameW")
)
// GetModuleFileName() with hModule = NULL
func executable() (exePath string, err error) {
return getModuleFileName()
}
func getModuleFileName() (string, error) {
var n uint32
b := make([]uint16, syscall.MAX_PATH)
size := uint32(len(b))
r0, _, e1 := getModuleFileNameProc.Call(0, uintptr(unsafe.Pointer(&b[0])), uintptr(size))
n = uint32(r0)
if n == 0 {
return "", e1
}
return string(utf16.Decode(b[0:n])), nil
}

View File

@@ -0,0 +1,35 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package bcrypt
import "encoding/base64"
const alphabet = "./ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
var bcEncoding = base64.NewEncoding(alphabet)
func base64Encode(src []byte) []byte {
n := bcEncoding.EncodedLen(len(src))
dst := make([]byte, n)
bcEncoding.Encode(dst, src)
for dst[n-1] == '=' {
n--
}
return dst[:n]
}
func base64Decode(src []byte) ([]byte, error) {
numOfEquals := 4 - (len(src) % 4)
for i := 0; i < numOfEquals; i++ {
src = append(src, '=')
}
dst := make([]byte, bcEncoding.DecodedLen(len(src)))
n, err := bcEncoding.Decode(dst, src)
if err != nil {
return nil, err
}
return dst[:n], nil
}

View File

@@ -0,0 +1,294 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package bcrypt implements Provos and Mazières's bcrypt adaptive hashing
// algorithm. See http://www.usenix.org/event/usenix99/provos/provos.pdf
package bcrypt
// The code is a port of Provos and Mazières's C implementation.
import (
"code.google.com/p/go.crypto/blowfish"
"crypto/rand"
"crypto/subtle"
"errors"
"fmt"
"io"
"strconv"
)
const (
MinCost int = 4 // the minimum allowable cost as passed in to GenerateFromPassword
MaxCost int = 31 // the maximum allowable cost as passed in to GenerateFromPassword
DefaultCost int = 10 // the cost that will actually be set if a cost below MinCost is passed into GenerateFromPassword
)
// The error returned from CompareHashAndPassword when a password and hash do
// not match.
var ErrMismatchedHashAndPassword = errors.New("crypto/bcrypt: hashedPassword is not the hash of the given password")
// The error returned from CompareHashAndPassword when a hash is too short to
// be a bcrypt hash.
var ErrHashTooShort = errors.New("crypto/bcrypt: hashedSecret too short to be a bcrypted password")
// The error returned from CompareHashAndPassword when a hash was created with
// a bcrypt algorithm newer than this implementation.
type HashVersionTooNewError byte
func (hv HashVersionTooNewError) Error() string {
return fmt.Sprintf("crypto/bcrypt: bcrypt algorithm version '%c' requested is newer than current version '%c'", byte(hv), majorVersion)
}
// The error returned from CompareHashAndPassword when a hash starts with something other than '$'
type InvalidHashPrefixError byte
func (ih InvalidHashPrefixError) Error() string {
return fmt.Sprintf("crypto/bcrypt: bcrypt hashes must start with '$', but hashedSecret started with '%c'", byte(ih))
}
type InvalidCostError int
func (ic InvalidCostError) Error() string {
return fmt.Sprintf("crypto/bcrypt: cost %d is outside allowed range (%d,%d)", int(ic), int(MinCost), int(MaxCost))
}
const (
majorVersion = '2'
minorVersion = 'a'
maxSaltSize = 16
maxCryptedHashSize = 23
encodedSaltSize = 22
encodedHashSize = 31
minHashSize = 59
)
// magicCipherData is an IV for the 64 Blowfish encryption calls in
// bcrypt(). It's the string "OrpheanBeholderScryDoubt" in big-endian bytes.
var magicCipherData = []byte{
0x4f, 0x72, 0x70, 0x68,
0x65, 0x61, 0x6e, 0x42,
0x65, 0x68, 0x6f, 0x6c,
0x64, 0x65, 0x72, 0x53,
0x63, 0x72, 0x79, 0x44,
0x6f, 0x75, 0x62, 0x74,
}
type hashed struct {
hash []byte
salt []byte
cost int // allowed range is MinCost to MaxCost
major byte
minor byte
}
// GenerateFromPassword returns the bcrypt hash of the password at the given
// cost. If the cost given is less than MinCost, the cost will be set to
// DefaultCost, instead. Use CompareHashAndPassword, as defined in this package,
// to compare the returned hashed password with its cleartext version.
func GenerateFromPassword(password []byte, cost int) ([]byte, error) {
p, err := newFromPassword(password, cost)
if err != nil {
return nil, err
}
return p.Hash(), nil
}
// CompareHashAndPassword compares a bcrypt hashed password with its possible
// plaintext equivalent. Returns nil on success, or an error on failure.
func CompareHashAndPassword(hashedPassword, password []byte) error {
p, err := newFromHash(hashedPassword)
if err != nil {
return err
}
otherHash, err := bcrypt(password, p.cost, p.salt)
if err != nil {
return err
}
otherP := &hashed{otherHash, p.salt, p.cost, p.major, p.minor}
if subtle.ConstantTimeCompare(p.Hash(), otherP.Hash()) == 1 {
return nil
}
return ErrMismatchedHashAndPassword
}
// Cost returns the hashing cost used to create the given hashed
// password. When, in the future, the hashing cost of a password system needs
// to be increased in order to adjust for greater computational power, this
// function allows one to establish which passwords need to be updated.
func Cost(hashedPassword []byte) (int, error) {
p, err := newFromHash(hashedPassword)
if err != nil {
return 0, err
}
return p.cost, nil
}
func newFromPassword(password []byte, cost int) (*hashed, error) {
if cost < MinCost {
cost = DefaultCost
}
p := new(hashed)
p.major = majorVersion
p.minor = minorVersion
err := checkCost(cost)
if err != nil {
return nil, err
}
p.cost = cost
unencodedSalt := make([]byte, maxSaltSize)
_, err = io.ReadFull(rand.Reader, unencodedSalt)
if err != nil {
return nil, err
}
p.salt = base64Encode(unencodedSalt)
hash, err := bcrypt(password, p.cost, p.salt)
if err != nil {
return nil, err
}
p.hash = hash
return p, err
}
func newFromHash(hashedSecret []byte) (*hashed, error) {
if len(hashedSecret) < minHashSize {
return nil, ErrHashTooShort
}
p := new(hashed)
n, err := p.decodeVersion(hashedSecret)
if err != nil {
return nil, err
}
hashedSecret = hashedSecret[n:]
n, err = p.decodeCost(hashedSecret)
if err != nil {
return nil, err
}
hashedSecret = hashedSecret[n:]
// The "+2" is here because we'll have to append at most 2 '=' to the salt
// when base64 decoding it in expensiveBlowfishSetup().
p.salt = make([]byte, encodedSaltSize, encodedSaltSize+2)
copy(p.salt, hashedSecret[:encodedSaltSize])
hashedSecret = hashedSecret[encodedSaltSize:]
p.hash = make([]byte, len(hashedSecret))
copy(p.hash, hashedSecret)
return p, nil
}
func bcrypt(password []byte, cost int, salt []byte) ([]byte, error) {
cipherData := make([]byte, len(magicCipherData))
copy(cipherData, magicCipherData)
c, err := expensiveBlowfishSetup(password, uint32(cost), salt)
if err != nil {
return nil, err
}
for i := 0; i < 24; i += 8 {
for j := 0; j < 64; j++ {
c.Encrypt(cipherData[i:i+8], cipherData[i:i+8])
}
}
// Bug compatibility with C bcrypt implementations. We only encode 23 of
// the 24 bytes encrypted.
hsh := base64Encode(cipherData[:maxCryptedHashSize])
return hsh, nil
}
func expensiveBlowfishSetup(key []byte, cost uint32, salt []byte) (*blowfish.Cipher, error) {
csalt, err := base64Decode(salt)
if err != nil {
return nil, err
}
// Bug compatibility with C bcrypt implementations. They use the trailing
// NULL in the key string during expansion.
ckey := append(key, 0)
c, err := blowfish.NewSaltedCipher(ckey, csalt)
if err != nil {
return nil, err
}
var i, rounds uint64
rounds = 1 << cost
for i = 0; i < rounds; i++ {
blowfish.ExpandKey(ckey, c)
blowfish.ExpandKey(csalt, c)
}
return c, nil
}
func (p *hashed) Hash() []byte {
arr := make([]byte, 60)
arr[0] = '$'
arr[1] = p.major
n := 2
if p.minor != 0 {
arr[2] = p.minor
n = 3
}
arr[n] = '$'
n += 1
copy(arr[n:], []byte(fmt.Sprintf("%02d", p.cost)))
n += 2
arr[n] = '$'
n += 1
copy(arr[n:], p.salt)
n += encodedSaltSize
copy(arr[n:], p.hash)
n += encodedHashSize
return arr[:n]
}
func (p *hashed) decodeVersion(sbytes []byte) (int, error) {
if sbytes[0] != '$' {
return -1, InvalidHashPrefixError(sbytes[0])
}
if sbytes[1] > majorVersion {
return -1, HashVersionTooNewError(sbytes[1])
}
p.major = sbytes[1]
n := 3
if sbytes[2] != '$' {
p.minor = sbytes[2]
n++
}
return n, nil
}
// sbytes should begin where decodeVersion left off.
func (p *hashed) decodeCost(sbytes []byte) (int, error) {
cost, err := strconv.Atoi(string(sbytes[0:2]))
if err != nil {
return -1, err
}
err = checkCost(cost)
if err != nil {
return -1, err
}
p.cost = cost
return 3, nil
}
func (p *hashed) String() string {
return fmt.Sprintf("&{hash: %#v, salt: %#v, cost: %d, major: %c, minor: %c}", string(p.hash), p.salt, p.cost, p.major, p.minor)
}
func checkCost(cost int) error {
if cost < MinCost || cost > MaxCost {
return InvalidCostError(cost)
}
return nil
}

View File

@@ -0,0 +1,217 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package bcrypt
import (
"bytes"
"fmt"
"testing"
)
func TestBcryptingIsEasy(t *testing.T) {
pass := []byte("mypassword")
hp, err := GenerateFromPassword(pass, 0)
if err != nil {
t.Fatalf("GenerateFromPassword error: %s", err)
}
if CompareHashAndPassword(hp, pass) != nil {
t.Errorf("%v should hash %s correctly", hp, pass)
}
notPass := "notthepass"
err = CompareHashAndPassword(hp, []byte(notPass))
if err != ErrMismatchedHashAndPassword {
t.Errorf("%v and %s should be mismatched", hp, notPass)
}
}
func TestBcryptingIsCorrect(t *testing.T) {
pass := []byte("allmine")
salt := []byte("XajjQvNhvvRt5GSeFk1xFe")
expectedHash := []byte("$2a$10$XajjQvNhvvRt5GSeFk1xFeyqRrsxkhBkUiQeg0dt.wU1qD4aFDcga")
hash, err := bcrypt(pass, 10, salt)
if err != nil {
t.Fatalf("bcrypt blew up: %v", err)
}
if !bytes.HasSuffix(expectedHash, hash) {
t.Errorf("%v should be the suffix of %v", hash, expectedHash)
}
h, err := newFromHash(expectedHash)
if err != nil {
t.Errorf("Unable to parse %s: %v", string(expectedHash), err)
}
// This is not the safe way to compare these hashes. We do this only for
// testing clarity. Use bcrypt.CompareHashAndPassword()
if err == nil && !bytes.Equal(expectedHash, h.Hash()) {
t.Errorf("Parsed hash %v should equal %v", h.Hash(), expectedHash)
}
}
func TestTooLongPasswordsWork(t *testing.T) {
salt := []byte("XajjQvNhvvRt5GSeFk1xFe")
// One byte over the usual 56 byte limit that blowfish has
tooLongPass := []byte("012345678901234567890123456789012345678901234567890123456")
tooLongExpected := []byte("$2a$10$XajjQvNhvvRt5GSeFk1xFe5l47dONXg781AmZtd869sO8zfsHuw7C")
hash, err := bcrypt(tooLongPass, 10, salt)
if err != nil {
t.Fatalf("bcrypt blew up on long password: %v", err)
}
if !bytes.HasSuffix(tooLongExpected, hash) {
t.Errorf("%v should be the suffix of %v", hash, tooLongExpected)
}
}
type InvalidHashTest struct {
err error
hash []byte
}
var invalidTests = []InvalidHashTest{
{ErrHashTooShort, []byte("$2a$10$fooo")},
{ErrHashTooShort, []byte("$2a")},
{HashVersionTooNewError('3'), []byte("$3a$10$sssssssssssssssssssssshhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh")},
{InvalidHashPrefixError('%'), []byte("%2a$10$sssssssssssssssssssssshhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh")},
{InvalidCostError(32), []byte("$2a$32$sssssssssssssssssssssshhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh")},
}
func TestInvalidHashErrors(t *testing.T) {
check := func(name string, expected, err error) {
if err == nil {
t.Errorf("%s: Should have returned an error", name)
}
if err != nil && err != expected {
t.Errorf("%s gave err %v but should have given %v", name, err, expected)
}
}
for _, iht := range invalidTests {
_, err := newFromHash(iht.hash)
check("newFromHash", iht.err, err)
err = CompareHashAndPassword(iht.hash, []byte("anything"))
check("CompareHashAndPassword", iht.err, err)
}
}
func TestUnpaddedBase64Encoding(t *testing.T) {
original := []byte{101, 201, 101, 75, 19, 227, 199, 20, 239, 236, 133, 32, 30, 109, 243, 30}
encodedOriginal := []byte("XajjQvNhvvRt5GSeFk1xFe")
encoded := base64Encode(original)
if !bytes.Equal(encodedOriginal, encoded) {
t.Errorf("Encoded %v should have equaled %v", encoded, encodedOriginal)
}
decoded, err := base64Decode(encodedOriginal)
if err != nil {
t.Fatalf("base64Decode blew up: %s", err)
}
if !bytes.Equal(decoded, original) {
t.Errorf("Decoded %v should have equaled %v", decoded, original)
}
}
func TestCost(t *testing.T) {
suffix := "XajjQvNhvvRt5GSeFk1xFe5l47dONXg781AmZtd869sO8zfsHuw7C"
for _, vers := range []string{"2a", "2"} {
for _, cost := range []int{4, 10} {
s := fmt.Sprintf("$%s$%02d$%s", vers, cost, suffix)
h := []byte(s)
actual, err := Cost(h)
if err != nil {
t.Errorf("Cost, error: %s", err)
continue
}
if actual != cost {
t.Errorf("Cost, expected: %d, actual: %d", cost, actual)
}
}
}
_, err := Cost([]byte("$a$a$" + suffix))
if err == nil {
t.Errorf("Cost, malformed but no error returned")
}
}
func TestCostValidationInHash(t *testing.T) {
if testing.Short() {
return
}
pass := []byte("mypassword")
for c := 0; c < MinCost; c++ {
p, _ := newFromPassword(pass, c)
if p.cost != DefaultCost {
t.Errorf("newFromPassword should default costs below %d to %d, but was %d", MinCost, DefaultCost, p.cost)
}
}
p, _ := newFromPassword(pass, 14)
if p.cost != 14 {
t.Errorf("newFromPassword should default cost to 14, but was %d", p.cost)
}
hp, _ := newFromHash(p.Hash())
if p.cost != hp.cost {
t.Errorf("newFromHash should maintain the cost at %d, but was %d", p.cost, hp.cost)
}
_, err := newFromPassword(pass, 32)
if err == nil {
t.Fatalf("newFromPassword: should return a cost error")
}
if err != InvalidCostError(32) {
t.Errorf("newFromPassword: should return cost error, got %#v", err)
}
}
func TestCostReturnsWithLeadingZeroes(t *testing.T) {
hp, _ := newFromPassword([]byte("abcdefgh"), 7)
cost := hp.Hash()[4:7]
expected := []byte("07$")
if !bytes.Equal(expected, cost) {
t.Errorf("single digit costs in hash should have leading zeros: was %v instead of %v", cost, expected)
}
}
func TestMinorNotRequired(t *testing.T) {
noMinorHash := []byte("$2$10$XajjQvNhvvRt5GSeFk1xFeyqRrsxkhBkUiQeg0dt.wU1qD4aFDcga")
h, err := newFromHash(noMinorHash)
if err != nil {
t.Fatalf("No minor hash blew up: %s", err)
}
if h.minor != 0 {
t.Errorf("Should leave minor version at 0, but was %d", h.minor)
}
if !bytes.Equal(noMinorHash, h.Hash()) {
t.Errorf("Should generate hash %v, but created %v", noMinorHash, h.Hash())
}
}
func BenchmarkEqual(b *testing.B) {
b.StopTimer()
passwd := []byte("somepasswordyoulike")
hash, _ := GenerateFromPassword(passwd, 10)
b.StartTimer()
for i := 0; i < b.N; i++ {
CompareHashAndPassword(hash, passwd)
}
}
func BenchmarkGeneration(b *testing.B) {
b.StopTimer()
passwd := []byte("mylongpassword1234")
b.StartTimer()
for i := 0; i < b.N; i++ {
GenerateFromPassword(passwd, 10)
}
}

View File

@@ -0,0 +1,190 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package blowfish
// ExpandKey performs a key expansion on the given *Cipher. Specifically, it
// performs the Blowfish algorithm's key schedule which sets up the *Cipher's
// pi and substitution tables for calls to Encrypt. This is used, primarily,
// by the bcrypt package to reuse the Blowfish key schedule during its
// set up. It's unlikely that you need to use this directly.
func ExpandKey(key []byte, c *Cipher) {
j := 0
for i := 0; i < 18; i++ {
var d uint32
for k := 0; k < 4; k++ {
d = d<<8 | uint32(key[j])
j++
if j >= len(key) {
j = 0
}
}
c.p[i] ^= d
}
var l, r uint32
for i := 0; i < 18; i += 2 {
l, r = encryptBlock(l, r, c)
c.p[i], c.p[i+1] = l, r
}
for i := 0; i < 256; i += 2 {
l, r = encryptBlock(l, r, c)
c.s0[i], c.s0[i+1] = l, r
}
for i := 0; i < 256; i += 2 {
l, r = encryptBlock(l, r, c)
c.s1[i], c.s1[i+1] = l, r
}
for i := 0; i < 256; i += 2 {
l, r = encryptBlock(l, r, c)
c.s2[i], c.s2[i+1] = l, r
}
for i := 0; i < 256; i += 2 {
l, r = encryptBlock(l, r, c)
c.s3[i], c.s3[i+1] = l, r
}
}
// This is similar to ExpandKey, but folds the salt during the key
// schedule. While ExpandKey is essentially expandKeyWithSalt with an all-zero
// salt passed in, reusing ExpandKey turns out to be a place of inefficiency
// and specializing it here is useful.
func expandKeyWithSalt(key []byte, salt []byte, c *Cipher) {
j := 0
for i := 0; i < 18; i++ {
var d uint32
for k := 0; k < 4; k++ {
d = d<<8 | uint32(key[j])
j++
if j >= len(key) {
j = 0
}
}
c.p[i] ^= d
}
j = 0
var expandedSalt [4]uint32
for i := range expandedSalt {
var d uint32
for k := 0; k < 4; k++ {
d = d<<8 | uint32(salt[j])
j++
if j >= len(salt) {
j = 0
}
}
expandedSalt[i] = d
}
var l, r uint32
for i := 0; i < 18; i += 2 {
l ^= expandedSalt[i&2]
r ^= expandedSalt[(i&2)+1]
l, r = encryptBlock(l, r, c)
c.p[i], c.p[i+1] = l, r
}
for i := 0; i < 256; i += 4 {
l ^= expandedSalt[2]
r ^= expandedSalt[3]
l, r = encryptBlock(l, r, c)
c.s0[i], c.s0[i+1] = l, r
l ^= expandedSalt[0]
r ^= expandedSalt[1]
l, r = encryptBlock(l, r, c)
c.s0[i+2], c.s0[i+3] = l, r
}
for i := 0; i < 256; i += 4 {
l ^= expandedSalt[2]
r ^= expandedSalt[3]
l, r = encryptBlock(l, r, c)
c.s1[i], c.s1[i+1] = l, r
l ^= expandedSalt[0]
r ^= expandedSalt[1]
l, r = encryptBlock(l, r, c)
c.s1[i+2], c.s1[i+3] = l, r
}
for i := 0; i < 256; i += 4 {
l ^= expandedSalt[2]
r ^= expandedSalt[3]
l, r = encryptBlock(l, r, c)
c.s2[i], c.s2[i+1] = l, r
l ^= expandedSalt[0]
r ^= expandedSalt[1]
l, r = encryptBlock(l, r, c)
c.s2[i+2], c.s2[i+3] = l, r
}
for i := 0; i < 256; i += 4 {
l ^= expandedSalt[2]
r ^= expandedSalt[3]
l, r = encryptBlock(l, r, c)
c.s3[i], c.s3[i+1] = l, r
l ^= expandedSalt[0]
r ^= expandedSalt[1]
l, r = encryptBlock(l, r, c)
c.s3[i+2], c.s3[i+3] = l, r
}
}
func encryptBlock(l, r uint32, c *Cipher) (uint32, uint32) {
xl, xr := l, r
xl ^= c.p[0]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[1]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[2]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[3]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[4]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[5]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[6]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[7]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[8]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[9]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[10]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[11]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[12]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[13]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[14]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[15]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[16]
xr ^= c.p[17]
return xr, xl
}
func decryptBlock(l, r uint32, c *Cipher) (uint32, uint32) {
xl, xr := l, r
xl ^= c.p[17]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[16]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[15]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[14]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[13]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[12]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[11]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[10]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[9]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[8]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[7]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[6]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[5]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[4]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[3]
xr ^= ((c.s0[byte(xl>>24)] + c.s1[byte(xl>>16)]) ^ c.s2[byte(xl>>8)]) + c.s3[byte(xl)] ^ c.p[2]
xl ^= ((c.s0[byte(xr>>24)] + c.s1[byte(xr>>16)]) ^ c.s2[byte(xr>>8)]) + c.s3[byte(xr)] ^ c.p[1]
xr ^= c.p[0]
return xr, xl
}
func zero(x []uint32) {
for i := range x {
x[i] = 0
}
}

View File

@@ -0,0 +1,210 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package blowfish
import (
"testing"
)
type CryptTest struct {
key []byte
in []byte
out []byte
}
// Test vector values are from http://www.schneier.com/code/vectors.txt.
var encryptTests = []CryptTest{
{
[]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
[]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
[]byte{0x4E, 0xF9, 0x97, 0x45, 0x61, 0x98, 0xDD, 0x78}},
{
[]byte{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF},
[]byte{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF},
[]byte{0x51, 0x86, 0x6F, 0xD5, 0xB8, 0x5E, 0xCB, 0x8A}},
{
[]byte{0x30, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
[]byte{0x10, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01},
[]byte{0x7D, 0x85, 0x6F, 0x9A, 0x61, 0x30, 0x63, 0xF2}},
{
[]byte{0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11},
[]byte{0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11},
[]byte{0x24, 0x66, 0xDD, 0x87, 0x8B, 0x96, 0x3C, 0x9D}},
{
[]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF},
[]byte{0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11},
[]byte{0x61, 0xF9, 0xC3, 0x80, 0x22, 0x81, 0xB0, 0x96}},
{
[]byte{0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11, 0x11},
[]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF},
[]byte{0x7D, 0x0C, 0xC6, 0x30, 0xAF, 0xDA, 0x1E, 0xC7}},
{
[]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
[]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
[]byte{0x4E, 0xF9, 0x97, 0x45, 0x61, 0x98, 0xDD, 0x78}},
{
[]byte{0xFE, 0xDC, 0xBA, 0x98, 0x76, 0x54, 0x32, 0x10},
[]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF},
[]byte{0x0A, 0xCE, 0xAB, 0x0F, 0xC6, 0xA0, 0xA2, 0x8D}},
{
[]byte{0x7C, 0xA1, 0x10, 0x45, 0x4A, 0x1A, 0x6E, 0x57},
[]byte{0x01, 0xA1, 0xD6, 0xD0, 0x39, 0x77, 0x67, 0x42},
[]byte{0x59, 0xC6, 0x82, 0x45, 0xEB, 0x05, 0x28, 0x2B}},
{
[]byte{0x01, 0x31, 0xD9, 0x61, 0x9D, 0xC1, 0x37, 0x6E},
[]byte{0x5C, 0xD5, 0x4C, 0xA8, 0x3D, 0xEF, 0x57, 0xDA},
[]byte{0xB1, 0xB8, 0xCC, 0x0B, 0x25, 0x0F, 0x09, 0xA0}},
{
[]byte{0x07, 0xA1, 0x13, 0x3E, 0x4A, 0x0B, 0x26, 0x86},
[]byte{0x02, 0x48, 0xD4, 0x38, 0x06, 0xF6, 0x71, 0x72},
[]byte{0x17, 0x30, 0xE5, 0x77, 0x8B, 0xEA, 0x1D, 0xA4}},
{
[]byte{0x38, 0x49, 0x67, 0x4C, 0x26, 0x02, 0x31, 0x9E},
[]byte{0x51, 0x45, 0x4B, 0x58, 0x2D, 0xDF, 0x44, 0x0A},
[]byte{0xA2, 0x5E, 0x78, 0x56, 0xCF, 0x26, 0x51, 0xEB}},
{
[]byte{0x04, 0xB9, 0x15, 0xBA, 0x43, 0xFE, 0xB5, 0xB6},
[]byte{0x42, 0xFD, 0x44, 0x30, 0x59, 0x57, 0x7F, 0xA2},
[]byte{0x35, 0x38, 0x82, 0xB1, 0x09, 0xCE, 0x8F, 0x1A}},
{
[]byte{0x01, 0x13, 0xB9, 0x70, 0xFD, 0x34, 0xF2, 0xCE},
[]byte{0x05, 0x9B, 0x5E, 0x08, 0x51, 0xCF, 0x14, 0x3A},
[]byte{0x48, 0xF4, 0xD0, 0x88, 0x4C, 0x37, 0x99, 0x18}},
{
[]byte{0x01, 0x70, 0xF1, 0x75, 0x46, 0x8F, 0xB5, 0xE6},
[]byte{0x07, 0x56, 0xD8, 0xE0, 0x77, 0x47, 0x61, 0xD2},
[]byte{0x43, 0x21, 0x93, 0xB7, 0x89, 0x51, 0xFC, 0x98}},
{
[]byte{0x43, 0x29, 0x7F, 0xAD, 0x38, 0xE3, 0x73, 0xFE},
[]byte{0x76, 0x25, 0x14, 0xB8, 0x29, 0xBF, 0x48, 0x6A},
[]byte{0x13, 0xF0, 0x41, 0x54, 0xD6, 0x9D, 0x1A, 0xE5}},
{
[]byte{0x07, 0xA7, 0x13, 0x70, 0x45, 0xDA, 0x2A, 0x16},
[]byte{0x3B, 0xDD, 0x11, 0x90, 0x49, 0x37, 0x28, 0x02},
[]byte{0x2E, 0xED, 0xDA, 0x93, 0xFF, 0xD3, 0x9C, 0x79}},
{
[]byte{0x04, 0x68, 0x91, 0x04, 0xC2, 0xFD, 0x3B, 0x2F},
[]byte{0x26, 0x95, 0x5F, 0x68, 0x35, 0xAF, 0x60, 0x9A},
[]byte{0xD8, 0x87, 0xE0, 0x39, 0x3C, 0x2D, 0xA6, 0xE3}},
{
[]byte{0x37, 0xD0, 0x6B, 0xB5, 0x16, 0xCB, 0x75, 0x46},
[]byte{0x16, 0x4D, 0x5E, 0x40, 0x4F, 0x27, 0x52, 0x32},
[]byte{0x5F, 0x99, 0xD0, 0x4F, 0x5B, 0x16, 0x39, 0x69}},
{
[]byte{0x1F, 0x08, 0x26, 0x0D, 0x1A, 0xC2, 0x46, 0x5E},
[]byte{0x6B, 0x05, 0x6E, 0x18, 0x75, 0x9F, 0x5C, 0xCA},
[]byte{0x4A, 0x05, 0x7A, 0x3B, 0x24, 0xD3, 0x97, 0x7B}},
{
[]byte{0x58, 0x40, 0x23, 0x64, 0x1A, 0xBA, 0x61, 0x76},
[]byte{0x00, 0x4B, 0xD6, 0xEF, 0x09, 0x17, 0x60, 0x62},
[]byte{0x45, 0x20, 0x31, 0xC1, 0xE4, 0xFA, 0xDA, 0x8E}},
{
[]byte{0x02, 0x58, 0x16, 0x16, 0x46, 0x29, 0xB0, 0x07},
[]byte{0x48, 0x0D, 0x39, 0x00, 0x6E, 0xE7, 0x62, 0xF2},
[]byte{0x75, 0x55, 0xAE, 0x39, 0xF5, 0x9B, 0x87, 0xBD}},
{
[]byte{0x49, 0x79, 0x3E, 0xBC, 0x79, 0xB3, 0x25, 0x8F},
[]byte{0x43, 0x75, 0x40, 0xC8, 0x69, 0x8F, 0x3C, 0xFA},
[]byte{0x53, 0xC5, 0x5F, 0x9C, 0xB4, 0x9F, 0xC0, 0x19}},
{
[]byte{0x4F, 0xB0, 0x5E, 0x15, 0x15, 0xAB, 0x73, 0xA7},
[]byte{0x07, 0x2D, 0x43, 0xA0, 0x77, 0x07, 0x52, 0x92},
[]byte{0x7A, 0x8E, 0x7B, 0xFA, 0x93, 0x7E, 0x89, 0xA3}},
{
[]byte{0x49, 0xE9, 0x5D, 0x6D, 0x4C, 0xA2, 0x29, 0xBF},
[]byte{0x02, 0xFE, 0x55, 0x77, 0x81, 0x17, 0xF1, 0x2A},
[]byte{0xCF, 0x9C, 0x5D, 0x7A, 0x49, 0x86, 0xAD, 0xB5}},
{
[]byte{0x01, 0x83, 0x10, 0xDC, 0x40, 0x9B, 0x26, 0xD6},
[]byte{0x1D, 0x9D, 0x5C, 0x50, 0x18, 0xF7, 0x28, 0xC2},
[]byte{0xD1, 0xAB, 0xB2, 0x90, 0x65, 0x8B, 0xC7, 0x78}},
{
[]byte{0x1C, 0x58, 0x7F, 0x1C, 0x13, 0x92, 0x4F, 0xEF},
[]byte{0x30, 0x55, 0x32, 0x28, 0x6D, 0x6F, 0x29, 0x5A},
[]byte{0x55, 0xCB, 0x37, 0x74, 0xD1, 0x3E, 0xF2, 0x01}},
{
[]byte{0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01, 0x01},
[]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF},
[]byte{0xFA, 0x34, 0xEC, 0x48, 0x47, 0xB2, 0x68, 0xB2}},
{
[]byte{0x1F, 0x1F, 0x1F, 0x1F, 0x0E, 0x0E, 0x0E, 0x0E},
[]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF},
[]byte{0xA7, 0x90, 0x79, 0x51, 0x08, 0xEA, 0x3C, 0xAE}},
{
[]byte{0xE0, 0xFE, 0xE0, 0xFE, 0xF1, 0xFE, 0xF1, 0xFE},
[]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF},
[]byte{0xC3, 0x9E, 0x07, 0x2D, 0x9F, 0xAC, 0x63, 0x1D}},
{
[]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
[]byte{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF},
[]byte{0x01, 0x49, 0x33, 0xE0, 0xCD, 0xAF, 0xF6, 0xE4}},
{
[]byte{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF},
[]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
[]byte{0xF2, 0x1E, 0x9A, 0x77, 0xB7, 0x1C, 0x49, 0xBC}},
{
[]byte{0x01, 0x23, 0x45, 0x67, 0x89, 0xAB, 0xCD, 0xEF},
[]byte{0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00},
[]byte{0x24, 0x59, 0x46, 0x88, 0x57, 0x54, 0x36, 0x9A}},
{
[]byte{0xFE, 0xDC, 0xBA, 0x98, 0x76, 0x54, 0x32, 0x10},
[]byte{0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF},
[]byte{0x6B, 0x5C, 0x5A, 0x9C, 0x5D, 0x9E, 0x0A, 0x5A}},
}
func TestCipherEncrypt(t *testing.T) {
for i, tt := range encryptTests {
c, err := NewCipher(tt.key)
if err != nil {
t.Errorf("NewCipher(%d bytes) = %s", len(tt.key), err)
continue
}
ct := make([]byte, len(tt.out))
c.Encrypt(ct, tt.in)
for j, v := range ct {
if v != tt.out[j] {
t.Errorf("Cipher.Encrypt, test vector #%d: cipher-text[%d] = %#x, expected %#x", i, j, v, tt.out[j])
break
}
}
}
}
func TestCipherDecrypt(t *testing.T) {
for i, tt := range encryptTests {
c, err := NewCipher(tt.key)
if err != nil {
t.Errorf("NewCipher(%d bytes) = %s", len(tt.key), err)
continue
}
pt := make([]byte, len(tt.in))
c.Decrypt(pt, tt.out)
for j, v := range pt {
if v != tt.in[j] {
t.Errorf("Cipher.Decrypt, test vector #%d: plain-text[%d] = %#x, expected %#x", i, j, v, tt.in[j])
break
}
}
}
}
func TestSaltedCipherKeyLength(t *testing.T) {
var key []byte
for i := 0; i < 4; i++ {
_, err := NewSaltedCipher(key, []byte{'a'})
if err != KeySizeError(i) {
t.Errorf("NewSaltedCipher with short key, gave error %#v, expected %#v", err, KeySizeError(i))
}
key = append(key, 'a')
}
// A 57-byte key. One over the typical blowfish restriction.
key = []byte("012345678901234567890123456789012345678901234567890123456")
_, err := NewSaltedCipher(key, []byte{'a'})
if err != nil {
t.Errorf("NewSaltedCipher with long key, gave error %#v", err)
}
}

View File

@@ -0,0 +1,90 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package blowfish implements Bruce Schneier's Blowfish encryption algorithm.
package blowfish
// The code is a port of Bruce Schneier's C implementation.
// See http://www.schneier.com/blowfish.html.
import "strconv"
// The Blowfish block size in bytes.
const BlockSize = 8
// A Cipher is an instance of Blowfish encryption using a particular key.
type Cipher struct {
p [18]uint32
s0, s1, s2, s3 [256]uint32
}
type KeySizeError int
func (k KeySizeError) Error() string {
return "crypto/blowfish: invalid key size " + strconv.Itoa(int(k))
}
// NewCipher creates and returns a Cipher.
// The key argument should be the Blowfish key, 4 to 56 bytes.
func NewCipher(key []byte) (*Cipher, error) {
var result Cipher
k := len(key)
if k < 4 || k > 56 {
return nil, KeySizeError(k)
}
initCipher(key, &result)
ExpandKey(key, &result)
return &result, nil
}
// NewSaltedCipher creates a returns a Cipher that folds a salt into its key
// schedule. For most purposes, NewCipher, instead of NewSaltedCipher, is
// sufficient and desirable. For bcrypt compatiblity, the key can be over 56
// bytes. Only the first 16 bytes of salt are used.
func NewSaltedCipher(key, salt []byte) (*Cipher, error) {
var result Cipher
k := len(key)
if k < 4 {
return nil, KeySizeError(k)
}
initCipher(key, &result)
expandKeyWithSalt(key, salt, &result)
return &result, nil
}
// BlockSize returns the Blowfish block size, 8 bytes.
// It is necessary to satisfy the Block interface in the
// package "crypto/cipher".
func (c *Cipher) BlockSize() int { return BlockSize }
// Encrypt encrypts the 8-byte buffer src using the key k
// and stores the result in dst.
// Note that for amounts of data larger than a block,
// it is not safe to just call Encrypt on successive blocks;
// instead, use an encryption mode like CBC (see crypto/cipher/cbc.go).
func (c *Cipher) Encrypt(dst, src []byte) {
l := uint32(src[0])<<24 | uint32(src[1])<<16 | uint32(src[2])<<8 | uint32(src[3])
r := uint32(src[4])<<24 | uint32(src[5])<<16 | uint32(src[6])<<8 | uint32(src[7])
l, r = encryptBlock(l, r, c)
dst[0], dst[1], dst[2], dst[3] = byte(l>>24), byte(l>>16), byte(l>>8), byte(l)
dst[4], dst[5], dst[6], dst[7] = byte(r>>24), byte(r>>16), byte(r>>8), byte(r)
}
// Decrypt decrypts the 8-byte buffer src using the key k
// and stores the result in dst.
func (c *Cipher) Decrypt(dst, src []byte) {
l := uint32(src[0])<<24 | uint32(src[1])<<16 | uint32(src[2])<<8 | uint32(src[3])
r := uint32(src[4])<<24 | uint32(src[5])<<16 | uint32(src[6])<<8 | uint32(src[7])
l, r = decryptBlock(l, r, c)
dst[0], dst[1], dst[2], dst[3] = byte(l>>24), byte(l>>16), byte(l>>8), byte(l)
dst[4], dst[5], dst[6], dst[7] = byte(r>>24), byte(r>>16), byte(r>>8), byte(r)
}
func initCipher(key []byte, c *Cipher) {
copy(c.p[0:], p[0:])
copy(c.s0[0:], s0[0:])
copy(c.s1[0:], s1[0:])
copy(c.s2[0:], s2[0:])
copy(c.s3[0:], s3[0:])
}

View File

@@ -0,0 +1,199 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// The startup permutation array and substitution boxes.
// They are the hexadecimal digits of PI; see:
// http://www.schneier.com/code/constants.txt.
package blowfish
var s0 = [256]uint32{
0xd1310ba6, 0x98dfb5ac, 0x2ffd72db, 0xd01adfb7, 0xb8e1afed, 0x6a267e96,
0xba7c9045, 0xf12c7f99, 0x24a19947, 0xb3916cf7, 0x0801f2e2, 0x858efc16,
0x636920d8, 0x71574e69, 0xa458fea3, 0xf4933d7e, 0x0d95748f, 0x728eb658,
0x718bcd58, 0x82154aee, 0x7b54a41d, 0xc25a59b5, 0x9c30d539, 0x2af26013,
0xc5d1b023, 0x286085f0, 0xca417918, 0xb8db38ef, 0x8e79dcb0, 0x603a180e,
0x6c9e0e8b, 0xb01e8a3e, 0xd71577c1, 0xbd314b27, 0x78af2fda, 0x55605c60,
0xe65525f3, 0xaa55ab94, 0x57489862, 0x63e81440, 0x55ca396a, 0x2aab10b6,
0xb4cc5c34, 0x1141e8ce, 0xa15486af, 0x7c72e993, 0xb3ee1411, 0x636fbc2a,
0x2ba9c55d, 0x741831f6, 0xce5c3e16, 0x9b87931e, 0xafd6ba33, 0x6c24cf5c,
0x7a325381, 0x28958677, 0x3b8f4898, 0x6b4bb9af, 0xc4bfe81b, 0x66282193,
0x61d809cc, 0xfb21a991, 0x487cac60, 0x5dec8032, 0xef845d5d, 0xe98575b1,
0xdc262302, 0xeb651b88, 0x23893e81, 0xd396acc5, 0x0f6d6ff3, 0x83f44239,
0x2e0b4482, 0xa4842004, 0x69c8f04a, 0x9e1f9b5e, 0x21c66842, 0xf6e96c9a,
0x670c9c61, 0xabd388f0, 0x6a51a0d2, 0xd8542f68, 0x960fa728, 0xab5133a3,
0x6eef0b6c, 0x137a3be4, 0xba3bf050, 0x7efb2a98, 0xa1f1651d, 0x39af0176,
0x66ca593e, 0x82430e88, 0x8cee8619, 0x456f9fb4, 0x7d84a5c3, 0x3b8b5ebe,
0xe06f75d8, 0x85c12073, 0x401a449f, 0x56c16aa6, 0x4ed3aa62, 0x363f7706,
0x1bfedf72, 0x429b023d, 0x37d0d724, 0xd00a1248, 0xdb0fead3, 0x49f1c09b,
0x075372c9, 0x80991b7b, 0x25d479d8, 0xf6e8def7, 0xe3fe501a, 0xb6794c3b,
0x976ce0bd, 0x04c006ba, 0xc1a94fb6, 0x409f60c4, 0x5e5c9ec2, 0x196a2463,
0x68fb6faf, 0x3e6c53b5, 0x1339b2eb, 0x3b52ec6f, 0x6dfc511f, 0x9b30952c,
0xcc814544, 0xaf5ebd09, 0xbee3d004, 0xde334afd, 0x660f2807, 0x192e4bb3,
0xc0cba857, 0x45c8740f, 0xd20b5f39, 0xb9d3fbdb, 0x5579c0bd, 0x1a60320a,
0xd6a100c6, 0x402c7279, 0x679f25fe, 0xfb1fa3cc, 0x8ea5e9f8, 0xdb3222f8,
0x3c7516df, 0xfd616b15, 0x2f501ec8, 0xad0552ab, 0x323db5fa, 0xfd238760,
0x53317b48, 0x3e00df82, 0x9e5c57bb, 0xca6f8ca0, 0x1a87562e, 0xdf1769db,
0xd542a8f6, 0x287effc3, 0xac6732c6, 0x8c4f5573, 0x695b27b0, 0xbbca58c8,
0xe1ffa35d, 0xb8f011a0, 0x10fa3d98, 0xfd2183b8, 0x4afcb56c, 0x2dd1d35b,
0x9a53e479, 0xb6f84565, 0xd28e49bc, 0x4bfb9790, 0xe1ddf2da, 0xa4cb7e33,
0x62fb1341, 0xcee4c6e8, 0xef20cada, 0x36774c01, 0xd07e9efe, 0x2bf11fb4,
0x95dbda4d, 0xae909198, 0xeaad8e71, 0x6b93d5a0, 0xd08ed1d0, 0xafc725e0,
0x8e3c5b2f, 0x8e7594b7, 0x8ff6e2fb, 0xf2122b64, 0x8888b812, 0x900df01c,
0x4fad5ea0, 0x688fc31c, 0xd1cff191, 0xb3a8c1ad, 0x2f2f2218, 0xbe0e1777,
0xea752dfe, 0x8b021fa1, 0xe5a0cc0f, 0xb56f74e8, 0x18acf3d6, 0xce89e299,
0xb4a84fe0, 0xfd13e0b7, 0x7cc43b81, 0xd2ada8d9, 0x165fa266, 0x80957705,
0x93cc7314, 0x211a1477, 0xe6ad2065, 0x77b5fa86, 0xc75442f5, 0xfb9d35cf,
0xebcdaf0c, 0x7b3e89a0, 0xd6411bd3, 0xae1e7e49, 0x00250e2d, 0x2071b35e,
0x226800bb, 0x57b8e0af, 0x2464369b, 0xf009b91e, 0x5563911d, 0x59dfa6aa,
0x78c14389, 0xd95a537f, 0x207d5ba2, 0x02e5b9c5, 0x83260376, 0x6295cfa9,
0x11c81968, 0x4e734a41, 0xb3472dca, 0x7b14a94a, 0x1b510052, 0x9a532915,
0xd60f573f, 0xbc9bc6e4, 0x2b60a476, 0x81e67400, 0x08ba6fb5, 0x571be91f,
0xf296ec6b, 0x2a0dd915, 0xb6636521, 0xe7b9f9b6, 0xff34052e, 0xc5855664,
0x53b02d5d, 0xa99f8fa1, 0x08ba4799, 0x6e85076a,
}
var s1 = [256]uint32{
0x4b7a70e9, 0xb5b32944, 0xdb75092e, 0xc4192623, 0xad6ea6b0, 0x49a7df7d,
0x9cee60b8, 0x8fedb266, 0xecaa8c71, 0x699a17ff, 0x5664526c, 0xc2b19ee1,
0x193602a5, 0x75094c29, 0xa0591340, 0xe4183a3e, 0x3f54989a, 0x5b429d65,
0x6b8fe4d6, 0x99f73fd6, 0xa1d29c07, 0xefe830f5, 0x4d2d38e6, 0xf0255dc1,
0x4cdd2086, 0x8470eb26, 0x6382e9c6, 0x021ecc5e, 0x09686b3f, 0x3ebaefc9,
0x3c971814, 0x6b6a70a1, 0x687f3584, 0x52a0e286, 0xb79c5305, 0xaa500737,
0x3e07841c, 0x7fdeae5c, 0x8e7d44ec, 0x5716f2b8, 0xb03ada37, 0xf0500c0d,
0xf01c1f04, 0x0200b3ff, 0xae0cf51a, 0x3cb574b2, 0x25837a58, 0xdc0921bd,
0xd19113f9, 0x7ca92ff6, 0x94324773, 0x22f54701, 0x3ae5e581, 0x37c2dadc,
0xc8b57634, 0x9af3dda7, 0xa9446146, 0x0fd0030e, 0xecc8c73e, 0xa4751e41,
0xe238cd99, 0x3bea0e2f, 0x3280bba1, 0x183eb331, 0x4e548b38, 0x4f6db908,
0x6f420d03, 0xf60a04bf, 0x2cb81290, 0x24977c79, 0x5679b072, 0xbcaf89af,
0xde9a771f, 0xd9930810, 0xb38bae12, 0xdccf3f2e, 0x5512721f, 0x2e6b7124,
0x501adde6, 0x9f84cd87, 0x7a584718, 0x7408da17, 0xbc9f9abc, 0xe94b7d8c,
0xec7aec3a, 0xdb851dfa, 0x63094366, 0xc464c3d2, 0xef1c1847, 0x3215d908,
0xdd433b37, 0x24c2ba16, 0x12a14d43, 0x2a65c451, 0x50940002, 0x133ae4dd,
0x71dff89e, 0x10314e55, 0x81ac77d6, 0x5f11199b, 0x043556f1, 0xd7a3c76b,
0x3c11183b, 0x5924a509, 0xf28fe6ed, 0x97f1fbfa, 0x9ebabf2c, 0x1e153c6e,
0x86e34570, 0xeae96fb1, 0x860e5e0a, 0x5a3e2ab3, 0x771fe71c, 0x4e3d06fa,
0x2965dcb9, 0x99e71d0f, 0x803e89d6, 0x5266c825, 0x2e4cc978, 0x9c10b36a,
0xc6150eba, 0x94e2ea78, 0xa5fc3c53, 0x1e0a2df4, 0xf2f74ea7, 0x361d2b3d,
0x1939260f, 0x19c27960, 0x5223a708, 0xf71312b6, 0xebadfe6e, 0xeac31f66,
0xe3bc4595, 0xa67bc883, 0xb17f37d1, 0x018cff28, 0xc332ddef, 0xbe6c5aa5,
0x65582185, 0x68ab9802, 0xeecea50f, 0xdb2f953b, 0x2aef7dad, 0x5b6e2f84,
0x1521b628, 0x29076170, 0xecdd4775, 0x619f1510, 0x13cca830, 0xeb61bd96,
0x0334fe1e, 0xaa0363cf, 0xb5735c90, 0x4c70a239, 0xd59e9e0b, 0xcbaade14,
0xeecc86bc, 0x60622ca7, 0x9cab5cab, 0xb2f3846e, 0x648b1eaf, 0x19bdf0ca,
0xa02369b9, 0x655abb50, 0x40685a32, 0x3c2ab4b3, 0x319ee9d5, 0xc021b8f7,
0x9b540b19, 0x875fa099, 0x95f7997e, 0x623d7da8, 0xf837889a, 0x97e32d77,
0x11ed935f, 0x16681281, 0x0e358829, 0xc7e61fd6, 0x96dedfa1, 0x7858ba99,
0x57f584a5, 0x1b227263, 0x9b83c3ff, 0x1ac24696, 0xcdb30aeb, 0x532e3054,
0x8fd948e4, 0x6dbc3128, 0x58ebf2ef, 0x34c6ffea, 0xfe28ed61, 0xee7c3c73,
0x5d4a14d9, 0xe864b7e3, 0x42105d14, 0x203e13e0, 0x45eee2b6, 0xa3aaabea,
0xdb6c4f15, 0xfacb4fd0, 0xc742f442, 0xef6abbb5, 0x654f3b1d, 0x41cd2105,
0xd81e799e, 0x86854dc7, 0xe44b476a, 0x3d816250, 0xcf62a1f2, 0x5b8d2646,
0xfc8883a0, 0xc1c7b6a3, 0x7f1524c3, 0x69cb7492, 0x47848a0b, 0x5692b285,
0x095bbf00, 0xad19489d, 0x1462b174, 0x23820e00, 0x58428d2a, 0x0c55f5ea,
0x1dadf43e, 0x233f7061, 0x3372f092, 0x8d937e41, 0xd65fecf1, 0x6c223bdb,
0x7cde3759, 0xcbee7460, 0x4085f2a7, 0xce77326e, 0xa6078084, 0x19f8509e,
0xe8efd855, 0x61d99735, 0xa969a7aa, 0xc50c06c2, 0x5a04abfc, 0x800bcadc,
0x9e447a2e, 0xc3453484, 0xfdd56705, 0x0e1e9ec9, 0xdb73dbd3, 0x105588cd,
0x675fda79, 0xe3674340, 0xc5c43465, 0x713e38d8, 0x3d28f89e, 0xf16dff20,
0x153e21e7, 0x8fb03d4a, 0xe6e39f2b, 0xdb83adf7,
}
var s2 = [256]uint32{
0xe93d5a68, 0x948140f7, 0xf64c261c, 0x94692934, 0x411520f7, 0x7602d4f7,
0xbcf46b2e, 0xd4a20068, 0xd4082471, 0x3320f46a, 0x43b7d4b7, 0x500061af,
0x1e39f62e, 0x97244546, 0x14214f74, 0xbf8b8840, 0x4d95fc1d, 0x96b591af,
0x70f4ddd3, 0x66a02f45, 0xbfbc09ec, 0x03bd9785, 0x7fac6dd0, 0x31cb8504,
0x96eb27b3, 0x55fd3941, 0xda2547e6, 0xabca0a9a, 0x28507825, 0x530429f4,
0x0a2c86da, 0xe9b66dfb, 0x68dc1462, 0xd7486900, 0x680ec0a4, 0x27a18dee,
0x4f3ffea2, 0xe887ad8c, 0xb58ce006, 0x7af4d6b6, 0xaace1e7c, 0xd3375fec,
0xce78a399, 0x406b2a42, 0x20fe9e35, 0xd9f385b9, 0xee39d7ab, 0x3b124e8b,
0x1dc9faf7, 0x4b6d1856, 0x26a36631, 0xeae397b2, 0x3a6efa74, 0xdd5b4332,
0x6841e7f7, 0xca7820fb, 0xfb0af54e, 0xd8feb397, 0x454056ac, 0xba489527,
0x55533a3a, 0x20838d87, 0xfe6ba9b7, 0xd096954b, 0x55a867bc, 0xa1159a58,
0xcca92963, 0x99e1db33, 0xa62a4a56, 0x3f3125f9, 0x5ef47e1c, 0x9029317c,
0xfdf8e802, 0x04272f70, 0x80bb155c, 0x05282ce3, 0x95c11548, 0xe4c66d22,
0x48c1133f, 0xc70f86dc, 0x07f9c9ee, 0x41041f0f, 0x404779a4, 0x5d886e17,
0x325f51eb, 0xd59bc0d1, 0xf2bcc18f, 0x41113564, 0x257b7834, 0x602a9c60,
0xdff8e8a3, 0x1f636c1b, 0x0e12b4c2, 0x02e1329e, 0xaf664fd1, 0xcad18115,
0x6b2395e0, 0x333e92e1, 0x3b240b62, 0xeebeb922, 0x85b2a20e, 0xe6ba0d99,
0xde720c8c, 0x2da2f728, 0xd0127845, 0x95b794fd, 0x647d0862, 0xe7ccf5f0,
0x5449a36f, 0x877d48fa, 0xc39dfd27, 0xf33e8d1e, 0x0a476341, 0x992eff74,
0x3a6f6eab, 0xf4f8fd37, 0xa812dc60, 0xa1ebddf8, 0x991be14c, 0xdb6e6b0d,
0xc67b5510, 0x6d672c37, 0x2765d43b, 0xdcd0e804, 0xf1290dc7, 0xcc00ffa3,
0xb5390f92, 0x690fed0b, 0x667b9ffb, 0xcedb7d9c, 0xa091cf0b, 0xd9155ea3,
0xbb132f88, 0x515bad24, 0x7b9479bf, 0x763bd6eb, 0x37392eb3, 0xcc115979,
0x8026e297, 0xf42e312d, 0x6842ada7, 0xc66a2b3b, 0x12754ccc, 0x782ef11c,
0x6a124237, 0xb79251e7, 0x06a1bbe6, 0x4bfb6350, 0x1a6b1018, 0x11caedfa,
0x3d25bdd8, 0xe2e1c3c9, 0x44421659, 0x0a121386, 0xd90cec6e, 0xd5abea2a,
0x64af674e, 0xda86a85f, 0xbebfe988, 0x64e4c3fe, 0x9dbc8057, 0xf0f7c086,
0x60787bf8, 0x6003604d, 0xd1fd8346, 0xf6381fb0, 0x7745ae04, 0xd736fccc,
0x83426b33, 0xf01eab71, 0xb0804187, 0x3c005e5f, 0x77a057be, 0xbde8ae24,
0x55464299, 0xbf582e61, 0x4e58f48f, 0xf2ddfda2, 0xf474ef38, 0x8789bdc2,
0x5366f9c3, 0xc8b38e74, 0xb475f255, 0x46fcd9b9, 0x7aeb2661, 0x8b1ddf84,
0x846a0e79, 0x915f95e2, 0x466e598e, 0x20b45770, 0x8cd55591, 0xc902de4c,
0xb90bace1, 0xbb8205d0, 0x11a86248, 0x7574a99e, 0xb77f19b6, 0xe0a9dc09,
0x662d09a1, 0xc4324633, 0xe85a1f02, 0x09f0be8c, 0x4a99a025, 0x1d6efe10,
0x1ab93d1d, 0x0ba5a4df, 0xa186f20f, 0x2868f169, 0xdcb7da83, 0x573906fe,
0xa1e2ce9b, 0x4fcd7f52, 0x50115e01, 0xa70683fa, 0xa002b5c4, 0x0de6d027,
0x9af88c27, 0x773f8641, 0xc3604c06, 0x61a806b5, 0xf0177a28, 0xc0f586e0,
0x006058aa, 0x30dc7d62, 0x11e69ed7, 0x2338ea63, 0x53c2dd94, 0xc2c21634,
0xbbcbee56, 0x90bcb6de, 0xebfc7da1, 0xce591d76, 0x6f05e409, 0x4b7c0188,
0x39720a3d, 0x7c927c24, 0x86e3725f, 0x724d9db9, 0x1ac15bb4, 0xd39eb8fc,
0xed545578, 0x08fca5b5, 0xd83d7cd3, 0x4dad0fc4, 0x1e50ef5e, 0xb161e6f8,
0xa28514d9, 0x6c51133c, 0x6fd5c7e7, 0x56e14ec4, 0x362abfce, 0xddc6c837,
0xd79a3234, 0x92638212, 0x670efa8e, 0x406000e0,
}
var s3 = [256]uint32{
0x3a39ce37, 0xd3faf5cf, 0xabc27737, 0x5ac52d1b, 0x5cb0679e, 0x4fa33742,
0xd3822740, 0x99bc9bbe, 0xd5118e9d, 0xbf0f7315, 0xd62d1c7e, 0xc700c47b,
0xb78c1b6b, 0x21a19045, 0xb26eb1be, 0x6a366eb4, 0x5748ab2f, 0xbc946e79,
0xc6a376d2, 0x6549c2c8, 0x530ff8ee, 0x468dde7d, 0xd5730a1d, 0x4cd04dc6,
0x2939bbdb, 0xa9ba4650, 0xac9526e8, 0xbe5ee304, 0xa1fad5f0, 0x6a2d519a,
0x63ef8ce2, 0x9a86ee22, 0xc089c2b8, 0x43242ef6, 0xa51e03aa, 0x9cf2d0a4,
0x83c061ba, 0x9be96a4d, 0x8fe51550, 0xba645bd6, 0x2826a2f9, 0xa73a3ae1,
0x4ba99586, 0xef5562e9, 0xc72fefd3, 0xf752f7da, 0x3f046f69, 0x77fa0a59,
0x80e4a915, 0x87b08601, 0x9b09e6ad, 0x3b3ee593, 0xe990fd5a, 0x9e34d797,
0x2cf0b7d9, 0x022b8b51, 0x96d5ac3a, 0x017da67d, 0xd1cf3ed6, 0x7c7d2d28,
0x1f9f25cf, 0xadf2b89b, 0x5ad6b472, 0x5a88f54c, 0xe029ac71, 0xe019a5e6,
0x47b0acfd, 0xed93fa9b, 0xe8d3c48d, 0x283b57cc, 0xf8d56629, 0x79132e28,
0x785f0191, 0xed756055, 0xf7960e44, 0xe3d35e8c, 0x15056dd4, 0x88f46dba,
0x03a16125, 0x0564f0bd, 0xc3eb9e15, 0x3c9057a2, 0x97271aec, 0xa93a072a,
0x1b3f6d9b, 0x1e6321f5, 0xf59c66fb, 0x26dcf319, 0x7533d928, 0xb155fdf5,
0x03563482, 0x8aba3cbb, 0x28517711, 0xc20ad9f8, 0xabcc5167, 0xccad925f,
0x4de81751, 0x3830dc8e, 0x379d5862, 0x9320f991, 0xea7a90c2, 0xfb3e7bce,
0x5121ce64, 0x774fbe32, 0xa8b6e37e, 0xc3293d46, 0x48de5369, 0x6413e680,
0xa2ae0810, 0xdd6db224, 0x69852dfd, 0x09072166, 0xb39a460a, 0x6445c0dd,
0x586cdecf, 0x1c20c8ae, 0x5bbef7dd, 0x1b588d40, 0xccd2017f, 0x6bb4e3bb,
0xdda26a7e, 0x3a59ff45, 0x3e350a44, 0xbcb4cdd5, 0x72eacea8, 0xfa6484bb,
0x8d6612ae, 0xbf3c6f47, 0xd29be463, 0x542f5d9e, 0xaec2771b, 0xf64e6370,
0x740e0d8d, 0xe75b1357, 0xf8721671, 0xaf537d5d, 0x4040cb08, 0x4eb4e2cc,
0x34d2466a, 0x0115af84, 0xe1b00428, 0x95983a1d, 0x06b89fb4, 0xce6ea048,
0x6f3f3b82, 0x3520ab82, 0x011a1d4b, 0x277227f8, 0x611560b1, 0xe7933fdc,
0xbb3a792b, 0x344525bd, 0xa08839e1, 0x51ce794b, 0x2f32c9b7, 0xa01fbac9,
0xe01cc87e, 0xbcc7d1f6, 0xcf0111c3, 0xa1e8aac7, 0x1a908749, 0xd44fbd9a,
0xd0dadecb, 0xd50ada38, 0x0339c32a, 0xc6913667, 0x8df9317c, 0xe0b12b4f,
0xf79e59b7, 0x43f5bb3a, 0xf2d519ff, 0x27d9459c, 0xbf97222c, 0x15e6fc2a,
0x0f91fc71, 0x9b941525, 0xfae59361, 0xceb69ceb, 0xc2a86459, 0x12baa8d1,
0xb6c1075e, 0xe3056a0c, 0x10d25065, 0xcb03a442, 0xe0ec6e0e, 0x1698db3b,
0x4c98a0be, 0x3278e964, 0x9f1f9532, 0xe0d392df, 0xd3a0342b, 0x8971f21e,
0x1b0a7441, 0x4ba3348c, 0xc5be7120, 0xc37632d8, 0xdf359f8d, 0x9b992f2e,
0xe60b6f47, 0x0fe3f11d, 0xe54cda54, 0x1edad891, 0xce6279cf, 0xcd3e7e6f,
0x1618b166, 0xfd2c1d05, 0x848fd2c5, 0xf6fb2299, 0xf523f357, 0xa6327623,
0x93a83531, 0x56cccd02, 0xacf08162, 0x5a75ebb5, 0x6e163697, 0x88d273cc,
0xde966292, 0x81b949d0, 0x4c50901b, 0x71c65614, 0xe6c6c7bd, 0x327a140a,
0x45e1d006, 0xc3f27b9a, 0xc9aa53fd, 0x62a80f00, 0xbb25bfe2, 0x35bdd2f6,
0x71126905, 0xb2040222, 0xb6cbcf7c, 0xcd769c2b, 0x53113ec0, 0x1640e3d3,
0x38abbd60, 0x2547adf0, 0xba38209c, 0xf746ce76, 0x77afa1c5, 0x20756060,
0x85cbfe4e, 0x8ae88dd8, 0x7aaaf9b0, 0x4cf9aa7e, 0x1948c25c, 0x02fb8a8c,
0x01c36ae4, 0xd6ebe1f9, 0x90d4f869, 0xa65cdea0, 0x3f09252d, 0xc208e69f,
0xb74e6132, 0xce77e25b, 0x578fdfe3, 0x3ac372e6,
}
var p = [18]uint32{
0x243f6a88, 0x85a308d3, 0x13198a2e, 0x03707344, 0xa4093822, 0x299f31d0,
0x082efa98, 0xec4e6c89, 0x452821e6, 0x38d01377, 0xbe5466cf, 0x34e90c6c,
0xc0ac29b7, 0xc97c50dd, 0x3f84d5b5, 0xb5470917, 0x9216d5d9, 0x8979fb1b,
}

View File

@@ -0,0 +1,37 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package transform_test
import (
"fmt"
"unicode"
"code.google.com/p/go.text/transform"
"code.google.com/p/go.text/unicode/norm"
)
func ExampleRemoveFunc() {
input := []byte(`tschüß; до свидания`)
b := make([]byte, len(input))
t := transform.RemoveFunc(unicode.IsSpace)
n, _, _ := t.Transform(b, input, true)
fmt.Println(string(b[:n]))
t = transform.RemoveFunc(func(r rune) bool {
return !unicode.Is(unicode.Latin, r)
})
n, _, _ = t.Transform(b, input, true)
fmt.Println(string(b[:n]))
n, _, _ = t.Transform(b, norm.NFD.Bytes(input), true)
fmt.Println(string(b[:n]))
// Output:
// tschüß;досвидания
// tschüß
// tschuß
}

View File

@@ -0,0 +1,496 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package transform provides reader and writer wrappers that transform the
// bytes passing through as well as various transformations. Example
// transformations provided by other packages include normalization and
// conversion between character sets.
package transform
import (
"errors"
"io"
"unicode/utf8"
)
var (
// ErrShortDst means that the destination buffer was too short to
// receive all of the transformed bytes.
ErrShortDst = errors.New("transform: short destination buffer")
// ErrShortSrc means that the source buffer has insufficient data to
// complete the transformation.
ErrShortSrc = errors.New("transform: short source buffer")
// errInconsistentByteCount means that Transform returned success (nil
// error) but also returned nSrc inconsistent with the src argument.
errInconsistentByteCount = errors.New("transform: inconsistent byte count returned")
// errShortInternal means that an internal buffer is not large enough
// to make progress and the Transform operation must be aborted.
errShortInternal = errors.New("transform: short internal buffer")
)
// Transformer transforms bytes.
type Transformer interface {
// Transform writes to dst the transformed bytes read from src, and
// returns the number of dst bytes written and src bytes read. The
// atEOF argument tells whether src represents the last bytes of the
// input.
//
// Callers should always process the nDst bytes produced and account
// for the nSrc bytes consumed before considering the error err.
//
// A nil error means that all of the transformed bytes (whether freshly
// transformed from src or left over from previous Transform calls)
// were written to dst. A nil error can be returned regardless of
// whether atEOF is true. If err is nil then nSrc must equal len(src);
// the converse is not necessarily true.
//
// ErrShortDst means that dst was too short to receive all of the
// transformed bytes. ErrShortSrc means that src had insufficient data
// to complete the transformation. If both conditions apply, then
// either error may be returned. Other than the error conditions listed
// here, implementations are free to report other errors that arise.
Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error)
}
// TODO: Do we require that a Transformer be reusable if it returns a nil error
// or do we always require a reset after use? Is Reset mandatory or optional?
// Reader wraps another io.Reader by transforming the bytes read.
type Reader struct {
r io.Reader
t Transformer
err error
// dst[dst0:dst1] contains bytes that have been transformed by t but
// not yet copied out via Read.
dst []byte
dst0, dst1 int
// src[src0:src1] contains bytes that have been read from r but not
// yet transformed through t.
src []byte
src0, src1 int
// transformComplete is whether the transformation is complete,
// regardless of whether or not it was successful.
transformComplete bool
}
const defaultBufSize = 4096
// NewReader returns a new Reader that wraps r by transforming the bytes read
// via t.
func NewReader(r io.Reader, t Transformer) *Reader {
return &Reader{
r: r,
t: t,
dst: make([]byte, defaultBufSize),
src: make([]byte, defaultBufSize),
}
}
// Read implements the io.Reader interface.
func (r *Reader) Read(p []byte) (int, error) {
n, err := 0, error(nil)
for {
// Copy out any transformed bytes and return the final error if we are done.
if r.dst0 != r.dst1 {
n = copy(p, r.dst[r.dst0:r.dst1])
r.dst0 += n
if r.dst0 == r.dst1 && r.transformComplete {
return n, r.err
}
return n, nil
} else if r.transformComplete {
return 0, r.err
}
// Try to transform some source bytes, or to flush the transformer if we
// are out of source bytes. We do this even if r.r.Read returned an error.
// As the io.Reader documentation says, "process the n > 0 bytes returned
// before considering the error".
if r.src0 != r.src1 || r.err != nil {
r.dst0 = 0
r.dst1, n, err = r.t.Transform(r.dst, r.src[r.src0:r.src1], r.err == io.EOF)
r.src0 += n
switch {
case err == nil:
if r.src0 != r.src1 {
r.err = errInconsistentByteCount
}
// The Transform call was successful; we are complete if we
// cannot read more bytes into src.
r.transformComplete = r.err != nil
continue
case err == ErrShortDst && r.dst1 != 0:
// Make room in dst by copying out, and try again.
continue
case err == ErrShortSrc && r.src1-r.src0 != len(r.src) && r.err == nil:
// Read more bytes into src via the code below, and try again.
default:
r.transformComplete = true
// The reader error (r.err) takes precedence over the
// transformer error (err) unless r.err is nil or io.EOF.
if r.err == nil || r.err == io.EOF {
r.err = err
}
continue
}
}
// Move any untransformed source bytes to the start of the buffer
// and read more bytes.
if r.src0 != 0 {
r.src0, r.src1 = 0, copy(r.src, r.src[r.src0:r.src1])
}
n, r.err = r.r.Read(r.src[r.src1:])
r.src1 += n
}
}
// TODO: implement ReadByte (and ReadRune??).
// Writer wraps another io.Writer by transforming the bytes read.
// The user needs to call Close to flush unwritten bytes that may
// be buffered.
type Writer struct {
w io.Writer
t Transformer
dst []byte
// src[:n] contains bytes that have not yet passed through t.
src []byte
n int
}
// NewWriter returns a new Writer that wraps w by transforming the bytes written
// via t.
func NewWriter(w io.Writer, t Transformer) *Writer {
return &Writer{
w: w,
t: t,
dst: make([]byte, defaultBufSize),
src: make([]byte, defaultBufSize),
}
}
// Write implements the io.Writer interface. If there are not enough
// bytes available to complete a Transform, the bytes will be buffered
// for the next write. Call Close to convert the remaining bytes.
func (w *Writer) Write(data []byte) (n int, err error) {
src := data
if w.n > 0 {
// Append bytes from data to the last remainder.
// TODO: limit the amount copied on first try.
n = copy(w.src[w.n:], data)
w.n += n
src = w.src[:w.n]
}
for {
nDst, nSrc, err := w.t.Transform(w.dst, src, false)
if _, werr := w.w.Write(w.dst[:nDst]); werr != nil {
return n, werr
}
src = src[nSrc:]
if w.n > 0 && len(src) <= n {
// Enough bytes from w.src have been consumed. We make src point
// to data instead to reduce the copying.
w.n = 0
n -= len(src)
src = data[n:]
if n < len(data) && (err == nil || err == ErrShortSrc) {
continue
}
} else {
n += nSrc
}
switch {
case err == ErrShortDst && nDst > 0:
case err == ErrShortSrc && len(src) < len(w.src):
m := copy(w.src, src)
// If w.n > 0, bytes from data were already copied to w.src and n
// was already set to the number of bytes consumed.
if w.n == 0 {
n += m
}
w.n = m
return n, nil
case err == nil && w.n > 0:
return n, errInconsistentByteCount
default:
return n, err
}
}
}
// Close implements the io.Closer interface.
func (w *Writer) Close() error {
for src := w.src[:w.n]; len(src) > 0; {
nDst, nSrc, err := w.t.Transform(w.dst, src, true)
if nDst == 0 {
return err
}
if _, werr := w.w.Write(w.dst[:nDst]); werr != nil {
return werr
}
if err != ErrShortDst {
return err
}
src = src[nSrc:]
}
return nil
}
type nop struct{}
func (nop) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
n := copy(dst, src)
if n < len(src) {
err = ErrShortDst
}
return n, n, err
}
type discard struct{}
func (discard) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
return 0, len(src), nil
}
var (
// Discard is a Transformer for which all Transform calls succeed
// by consuming all bytes and writing nothing.
Discard Transformer = discard{}
// Nop is a Transformer that copies src to dst.
Nop Transformer = nop{}
)
// chain is a sequence of links. A chain with N Transformers has N+1 links and
// N+1 buffers. Of those N+1 buffers, the first and last are the src and dst
// buffers given to chain.Transform and the middle N-1 buffers are intermediate
// buffers owned by the chain. The i'th link transforms bytes from the i'th
// buffer chain.link[i].b at read offset chain.link[i].p to the i+1'th buffer
// chain.link[i+1].b at write offset chain.link[i+1].n, for i in [0, N).
type chain struct {
link []link
err error
// errStart is the index at which the error occurred plus 1. Processing
// errStart at this level at the next call to Transform. As long as
// errStart > 0, chain will not consume any more source bytes.
errStart int
}
func (c *chain) fatalError(errIndex int, err error) {
if i := errIndex + 1; i > c.errStart {
c.errStart = i
c.err = err
}
}
type link struct {
t Transformer
// b[p:n] holds the bytes to be transformed by t.
b []byte
p int
n int
}
func (l *link) src() []byte {
return l.b[l.p:l.n]
}
func (l *link) dst() []byte {
return l.b[l.n:]
}
// Chain returns a Transformer that applies t in sequence.
func Chain(t ...Transformer) Transformer {
if len(t) == 0 {
return nop{}
}
c := &chain{link: make([]link, len(t)+1)}
for i, tt := range t {
c.link[i].t = tt
}
// Allocate intermediate buffers.
b := make([][defaultBufSize]byte, len(t)-1)
for i := range b {
c.link[i+1].b = b[i][:]
}
return c
}
// Transform applies the transformers of c in sequence.
func (c *chain) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
// Set up src and dst in the chain.
srcL := &c.link[0]
dstL := &c.link[len(c.link)-1]
srcL.b, srcL.p, srcL.n = src, 0, len(src)
dstL.b, dstL.n = dst, 0
var lastFull, needProgress bool // for detecting progress
// i is the index of the next Transformer to apply, for i in [low, high].
// low is the lowest index for which c.link[low] may still produce bytes.
// high is the highest index for which c.link[high] has a Transformer.
// The error returned by Transform determines whether to increase or
// decrease i. We try to completely fill a buffer before converting it.
for low, i, high := c.errStart, c.errStart, len(c.link)-2; low <= i && i <= high; {
in, out := &c.link[i], &c.link[i+1]
nDst, nSrc, err0 := in.t.Transform(out.dst(), in.src(), atEOF && low == i)
out.n += nDst
in.p += nSrc
if i > 0 && in.p == in.n {
in.p, in.n = 0, 0
}
needProgress, lastFull = lastFull, false
switch err0 {
case ErrShortDst:
// Process the destination buffer next. Return if we are already
// at the high index.
if i == high {
return dstL.n, srcL.p, ErrShortDst
}
if out.n != 0 {
i++
// If the Transformer at the next index is not able to process any
// source bytes there is nothing that can be done to make progress
// and the bytes will remain unprocessed. lastFull is used to
// detect this and break out of the loop with a fatal error.
lastFull = true
continue
}
// The destination buffer was too small, but is completely empty.
// Return a fatal error as this transformation can never complete.
c.fatalError(i, errShortInternal)
case ErrShortSrc:
if i == 0 {
// Save ErrShortSrc in err. All other errors take precedence.
err = ErrShortSrc
break
}
// Source bytes were depleted before filling up the destination buffer.
// Verify we made some progress, move the remaining bytes to the errStart
// and try to get more source bytes.
if needProgress && nSrc == 0 || in.n-in.p == len(in.b) {
// There were not enough source bytes to proceed while the source
// buffer cannot hold any more bytes. Return a fatal error as this
// transformation can never complete.
c.fatalError(i, errShortInternal)
break
}
// in.b is an internal buffer and we can make progress.
in.p, in.n = 0, copy(in.b, in.src())
fallthrough
case nil:
// if i == low, we have depleted the bytes at index i or any lower levels.
// In that case we increase low and i. In all other cases we decrease i to
// fetch more bytes before proceeding to the next index.
if i > low {
i--
continue
}
default:
c.fatalError(i, err0)
}
// Exhausted level low or fatal error: increase low and continue
// to process the bytes accepted so far.
i++
low = i
}
// If c.errStart > 0, this means we found a fatal error. We will clear
// all upstream buffers. At this point, no more progress can be made
// downstream, as Transform would have bailed while handling ErrShortDst.
if c.errStart > 0 {
for i := 1; i < c.errStart; i++ {
c.link[i].p, c.link[i].n = 0, 0
}
err, c.errStart, c.err = c.err, 0, nil
}
return dstL.n, srcL.p, err
}
// RemoveFunc returns a Transformer that removes from the input all runes r for
// which f(r) is true. Illegal bytes in the input are replaced by RuneError.
func RemoveFunc(f func(r rune) bool) Transformer {
return removeF(f)
}
type removeF func(r rune) bool
// Transform implements the Transformer interface.
func (t removeF) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
for r, sz := rune(0), 0; len(src) > 0; src = src[sz:] {
if r = rune(src[0]); r < utf8.RuneSelf {
sz = 1
} else {
r, sz = utf8.DecodeRune(src)
if sz == 1 {
// Invalid rune.
if !atEOF && !utf8.FullRune(src[nSrc:]) {
err = ErrShortSrc
break
}
// We replace illegal bytes with RuneError. Not doing so might
// otherwise turn a sequence of invalid UTF-8 into valid UTF-8.
// The resulting byte sequence may subsequently contain runes
// for which t(r) is true that were passed unnoticed.
if !t(r) {
if nDst+3 > len(dst) {
err = ErrShortDst
break
}
nDst += copy(dst[nDst:], "\uFFFD")
}
nSrc++
continue
}
}
if !t(r) {
if nDst+sz > len(dst) {
err = ErrShortDst
break
}
nDst += copy(dst[nDst:], src[:sz])
}
nSrc += sz
}
return
}
// Bytes returns a new byte slice with the result of converting b using t.
// If any unrecoverable error occurs it returns nil.
func Bytes(t Transformer, b []byte) []byte {
out := make([]byte, len(b))
n := 0
for {
nDst, nSrc, err := t.Transform(out[n:], b, true)
n += nDst
if err == nil {
return out[:n]
} else if err != ErrShortDst {
return nil
}
b = b[nSrc:]
// Grow the destination buffer.
sz := len(out)
if sz <= 256 {
sz *= 2
} else {
sz += sz >> 1
}
out2 := make([]byte, sz)
copy(out2, out[:n])
out = out2
}
}

View File

@@ -0,0 +1,901 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package transform
import (
"bytes"
"errors"
"fmt"
"io/ioutil"
"strconv"
"strings"
"testing"
"unicode/utf8"
)
type lowerCaseASCII struct{}
func (lowerCaseASCII) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
n := len(src)
if n > len(dst) {
n, err = len(dst), ErrShortDst
}
for i, c := range src[:n] {
if 'A' <= c && c <= 'Z' {
c += 'a' - 'A'
}
dst[i] = c
}
return n, n, err
}
var errYouMentionedX = errors.New("you mentioned X")
type dontMentionX struct{}
func (dontMentionX) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
n := len(src)
if n > len(dst) {
n, err = len(dst), ErrShortDst
}
for i, c := range src[:n] {
if c == 'X' {
return i, i, errYouMentionedX
}
dst[i] = c
}
return n, n, err
}
// doublerAtEOF is a strange Transformer that transforms "this" to "tthhiiss",
// but only if atEOF is true.
type doublerAtEOF struct{}
func (doublerAtEOF) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
if !atEOF {
return 0, 0, ErrShortSrc
}
for i, c := range src {
if 2*i+2 >= len(dst) {
return 2 * i, i, ErrShortDst
}
dst[2*i+0] = c
dst[2*i+1] = c
}
return 2 * len(src), len(src), nil
}
// rleDecode and rleEncode implement a toy run-length encoding: "aabbbbbbbbbb"
// is encoded as "2a10b". The decoding is assumed to not contain any numbers.
type rleDecode struct{}
func (rleDecode) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
loop:
for len(src) > 0 {
n := 0
for i, c := range src {
if '0' <= c && c <= '9' {
n = 10*n + int(c-'0')
continue
}
if i == 0 {
return nDst, nSrc, errors.New("rleDecode: bad input")
}
if n > len(dst) {
return nDst, nSrc, ErrShortDst
}
for j := 0; j < n; j++ {
dst[j] = c
}
dst, src = dst[n:], src[i+1:]
nDst, nSrc = nDst+n, nSrc+i+1
continue loop
}
if atEOF {
return nDst, nSrc, errors.New("rleDecode: bad input")
}
return nDst, nSrc, ErrShortSrc
}
return nDst, nSrc, nil
}
type rleEncode struct {
// allowStutter means that "xxxxxxxx" can be encoded as "5x3x"
// instead of always as "8x".
allowStutter bool
}
func (e rleEncode) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
for len(src) > 0 {
n, c0 := len(src), src[0]
for i, c := range src[1:] {
if c != c0 {
n = i + 1
break
}
}
if n == len(src) && !atEOF && !e.allowStutter {
return nDst, nSrc, ErrShortSrc
}
s := strconv.Itoa(n)
if len(s) >= len(dst) {
return nDst, nSrc, ErrShortDst
}
copy(dst, s)
dst[len(s)] = c0
dst, src = dst[len(s)+1:], src[n:]
nDst, nSrc = nDst+len(s)+1, nSrc+n
}
return nDst, nSrc, nil
}
type testCase struct {
desc string
t Transformer
src string
dstSize int
srcSize int
ioSize int
wantStr string
wantErr error
wantIter int // number of iterations taken; 0 means we don't care.
}
func (t testCase) String() string {
return tstr(t.t) + "; " + t.desc
}
func tstr(t Transformer) string {
if stringer, ok := t.(fmt.Stringer); ok {
return stringer.String()
}
s := fmt.Sprintf("%T", t)
return s[1+strings.Index(s, "."):]
}
func (c chain) String() string {
buf := &bytes.Buffer{}
buf.WriteString("Chain(")
for i, l := range c.link[:len(c.link)-1] {
if i != 0 {
fmt.Fprint(buf, ", ")
}
buf.WriteString(tstr(l.t))
}
buf.WriteString(")")
return buf.String()
}
var testCases = []testCase{
{
desc: "basic",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 100,
srcSize: 100,
wantStr: "hello world.",
},
{
desc: "small dst",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 3,
srcSize: 100,
wantStr: "hello world.",
},
{
desc: "small src",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 100,
srcSize: 4,
wantStr: "hello world.",
},
{
desc: "small buffers",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 3,
srcSize: 4,
wantStr: "hello world.",
},
{
desc: "very small buffers",
t: lowerCaseASCII{},
src: "Hello WORLD.",
dstSize: 1,
srcSize: 1,
wantStr: "hello world.",
},
{
desc: "basic",
t: dontMentionX{},
src: "The First Rule of Transform Club: don't mention Mister X, ever.",
dstSize: 100,
srcSize: 100,
wantStr: "The First Rule of Transform Club: don't mention Mister ",
wantErr: errYouMentionedX,
},
{
desc: "small buffers",
t: dontMentionX{},
src: "The First Rule of Transform Club: don't mention Mister X, ever.",
dstSize: 10,
srcSize: 10,
wantStr: "The First Rule of Transform Club: don't mention Mister ",
wantErr: errYouMentionedX,
},
{
desc: "very small buffers",
t: dontMentionX{},
src: "The First Rule of Transform Club: don't mention Mister X, ever.",
dstSize: 1,
srcSize: 1,
wantStr: "The First Rule of Transform Club: don't mention Mister ",
wantErr: errYouMentionedX,
},
{
desc: "only transform at EOF",
t: doublerAtEOF{},
src: "this",
dstSize: 100,
srcSize: 100,
wantStr: "tthhiiss",
},
{
desc: "basic",
t: rleDecode{},
src: "1a2b3c10d11e0f1g",
dstSize: 100,
srcSize: 100,
wantStr: "abbcccddddddddddeeeeeeeeeeeg",
},
{
desc: "long",
t: rleDecode{},
src: "12a23b34c45d56e99z",
dstSize: 100,
srcSize: 100,
wantStr: strings.Repeat("a", 12) +
strings.Repeat("b", 23) +
strings.Repeat("c", 34) +
strings.Repeat("d", 45) +
strings.Repeat("e", 56) +
strings.Repeat("z", 99),
},
{
desc: "tight buffers",
t: rleDecode{},
src: "1a2b3c10d11e0f1g",
dstSize: 11,
srcSize: 3,
wantStr: "abbcccddddddddddeeeeeeeeeeeg",
},
{
desc: "short dst",
t: rleDecode{},
src: "1a2b3c10d11e0f1g",
dstSize: 10,
srcSize: 3,
wantStr: "abbcccdddddddddd",
wantErr: ErrShortDst,
},
{
desc: "short src",
t: rleDecode{},
src: "1a2b3c10d11e0f1g",
dstSize: 11,
srcSize: 2,
ioSize: 2,
wantStr: "abbccc",
wantErr: ErrShortSrc,
},
{
desc: "basic",
t: rleEncode{},
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 100,
srcSize: 100,
wantStr: "1a2b3c10d11e1g",
},
{
desc: "long",
t: rleEncode{},
src: strings.Repeat("a", 12) +
strings.Repeat("b", 23) +
strings.Repeat("c", 34) +
strings.Repeat("d", 45) +
strings.Repeat("e", 56) +
strings.Repeat("z", 99),
dstSize: 100,
srcSize: 100,
wantStr: "12a23b34c45d56e99z",
},
{
desc: "tight buffers",
t: rleEncode{},
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 3,
srcSize: 12,
wantStr: "1a2b3c10d11e1g",
},
{
desc: "short dst",
t: rleEncode{},
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 2,
srcSize: 12,
wantStr: "1a2b3c",
wantErr: ErrShortDst,
},
{
desc: "short src",
t: rleEncode{},
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 3,
srcSize: 11,
ioSize: 11,
wantStr: "1a2b3c10d",
wantErr: ErrShortSrc,
},
{
desc: "allowStutter = false",
t: rleEncode{allowStutter: false},
src: "aaaabbbbbbbbccccddddd",
dstSize: 10,
srcSize: 10,
wantStr: "4a8b4c5d",
},
{
desc: "allowStutter = true",
t: rleEncode{allowStutter: true},
src: "aaaabbbbbbbbccccddddd",
dstSize: 10,
srcSize: 10,
ioSize: 10,
wantStr: "4a6b2b4c4d1d",
},
}
func TestReader(t *testing.T) {
for _, tc := range testCases {
reset(tc.t)
r := NewReader(strings.NewReader(tc.src), tc.t)
// Differently sized dst and src buffers are not part of the
// exported API. We override them manually.
r.dst = make([]byte, tc.dstSize)
r.src = make([]byte, tc.srcSize)
got, err := ioutil.ReadAll(r)
str := string(got)
if str != tc.wantStr || err != tc.wantErr {
t.Errorf("%s:\ngot %q, %v\nwant %q, %v", tc, str, err, tc.wantStr, tc.wantErr)
}
}
}
func reset(t Transformer) {
var dst [128]byte
for err := ErrShortDst; err != nil; {
_, _, err = t.Transform(dst[:], nil, true)
}
}
func TestWriter(t *testing.T) {
tests := append(testCases, chainTests()...)
for _, tc := range tests {
sizes := []int{1, 2, 3, 4, 5, 10, 100, 1000}
if tc.ioSize > 0 {
sizes = []int{tc.ioSize}
}
for _, sz := range sizes {
bb := &bytes.Buffer{}
reset(tc.t)
w := NewWriter(bb, tc.t)
// Differently sized dst and src buffers are not part of the
// exported API. We override them manually.
w.dst = make([]byte, tc.dstSize)
w.src = make([]byte, tc.srcSize)
src := make([]byte, sz)
var err error
for b := tc.src; len(b) > 0 && err == nil; {
n := copy(src, b)
b = b[n:]
m := 0
m, err = w.Write(src[:n])
if m != n && err == nil {
t.Errorf("%s:%d: did not consume all bytes %d < %d", tc, sz, m, n)
}
}
if err == nil {
err = w.Close()
}
str := bb.String()
if str != tc.wantStr || err != tc.wantErr {
t.Errorf("%s:%d:\ngot %q, %v\nwant %q, %v", tc, sz, str, err, tc.wantStr, tc.wantErr)
}
}
}
}
func TestNop(t *testing.T) {
testCases := []struct {
str string
dstSize int
err error
}{
{"", 0, nil},
{"", 10, nil},
{"a", 0, ErrShortDst},
{"a", 1, nil},
{"a", 10, nil},
}
for i, tc := range testCases {
dst := make([]byte, tc.dstSize)
nDst, nSrc, err := Nop.Transform(dst, []byte(tc.str), true)
want := tc.str
if tc.dstSize < len(want) {
want = want[:tc.dstSize]
}
if got := string(dst[:nDst]); got != want || err != tc.err || nSrc != nDst {
t.Errorf("%d:\ngot %q, %d, %v\nwant %q, %d, %v", i, got, nSrc, err, want, nDst, tc.err)
}
}
}
func TestDiscard(t *testing.T) {
testCases := []struct {
str string
dstSize int
}{
{"", 0},
{"", 10},
{"a", 0},
{"ab", 10},
}
for i, tc := range testCases {
nDst, nSrc, err := Discard.Transform(make([]byte, tc.dstSize), []byte(tc.str), true)
if nDst != 0 || nSrc != len(tc.str) || err != nil {
t.Errorf("%d:\ngot %q, %d, %v\nwant 0, %d, nil", i, nDst, nSrc, err, len(tc.str))
}
}
}
// mkChain creates a Chain transformer. x must be alternating between transformer
// and bufSize, like T, (sz, T)*
func mkChain(x ...interface{}) *chain {
t := []Transformer{}
for i := 0; i < len(x); i += 2 {
t = append(t, x[i].(Transformer))
}
c := Chain(t...).(*chain)
for i, j := 1, 1; i < len(x); i, j = i+2, j+1 {
c.link[j].b = make([]byte, x[i].(int))
}
return c
}
func chainTests() []testCase {
return []testCase{
{
desc: "nil error",
t: mkChain(rleEncode{}, 100, lowerCaseASCII{}),
src: "ABB",
dstSize: 100,
srcSize: 100,
wantStr: "1a2b",
wantErr: nil,
wantIter: 1,
},
{
desc: "short dst buffer",
t: mkChain(lowerCaseASCII{}, 3, rleDecode{}),
src: "1a2b3c10d11e0f1g",
dstSize: 10,
srcSize: 3,
wantStr: "abbcccdddddddddd",
wantErr: ErrShortDst,
},
{
desc: "short internal dst buffer",
t: mkChain(lowerCaseASCII{}, 3, rleDecode{}, 10, Nop),
src: "1a2b3c10d11e0f1g",
dstSize: 100,
srcSize: 3,
wantStr: "abbcccdddddddddd",
wantErr: errShortInternal,
},
{
desc: "short internal dst buffer from input",
t: mkChain(rleDecode{}, 10, Nop),
src: "1a2b3c10d11e0f1g",
dstSize: 100,
srcSize: 3,
wantStr: "abbcccdddddddddd",
wantErr: errShortInternal,
},
{
desc: "empty short internal dst buffer",
t: mkChain(lowerCaseASCII{}, 3, rleDecode{}, 10, Nop),
src: "4a7b11e0f1g",
dstSize: 100,
srcSize: 3,
wantStr: "aaaabbbbbbb",
wantErr: errShortInternal,
},
{
desc: "empty short internal dst buffer from input",
t: mkChain(rleDecode{}, 10, Nop),
src: "4a7b11e0f1g",
dstSize: 100,
srcSize: 3,
wantStr: "aaaabbbbbbb",
wantErr: errShortInternal,
},
{
desc: "short internal src buffer after full dst buffer",
t: mkChain(Nop, 5, rleEncode{}, 10, Nop),
src: "cccccddddd",
dstSize: 100,
srcSize: 100,
wantStr: "",
wantErr: errShortInternal,
wantIter: 1,
},
{
desc: "short internal src buffer after short dst buffer; test lastFull",
t: mkChain(rleDecode{}, 5, rleEncode{}, 4, Nop),
src: "2a1b4c6d",
dstSize: 100,
srcSize: 100,
wantStr: "2a1b",
wantErr: errShortInternal,
},
{
desc: "short internal src buffer after successful complete fill",
t: mkChain(Nop, 3, rleDecode{}),
src: "123a4b",
dstSize: 4,
srcSize: 3,
wantStr: "",
wantErr: errShortInternal,
wantIter: 1,
},
{
desc: "short internal src buffer after short dst buffer; test lastFull",
t: mkChain(rleDecode{}, 5, rleEncode{}),
src: "2a1b4c6d",
dstSize: 4,
srcSize: 100,
wantStr: "2a1b",
wantErr: errShortInternal,
},
{
desc: "short src buffer",
t: mkChain(rleEncode{}, 5, Nop),
src: "abbcccddddeeeee",
dstSize: 4,
srcSize: 4,
ioSize: 4,
wantStr: "1a2b3c",
wantErr: ErrShortSrc,
},
{
desc: "process all in one go",
t: mkChain(rleEncode{}, 5, Nop),
src: "abbcccddddeeeeeffffff",
dstSize: 100,
srcSize: 100,
wantStr: "1a2b3c4d5e6f",
wantErr: nil,
wantIter: 1,
},
{
desc: "complete processing downstream after error",
t: mkChain(dontMentionX{}, 2, rleDecode{}, 5, Nop),
src: "3a4b5eX",
dstSize: 100,
srcSize: 100,
ioSize: 100,
wantStr: "aaabbbbeeeee",
wantErr: errYouMentionedX,
},
{
desc: "return downstream fatal errors first (followed by short dst)",
t: mkChain(dontMentionX{}, 8, rleDecode{}, 4, Nop),
src: "3a4b5eX",
dstSize: 100,
srcSize: 100,
ioSize: 100,
wantStr: "aaabbbb",
wantErr: errShortInternal,
},
{
desc: "return downstream fatal errors first (followed by short src)",
t: mkChain(dontMentionX{}, 5, Nop, 1, rleDecode{}),
src: "1a5bX",
dstSize: 100,
srcSize: 100,
ioSize: 100,
wantStr: "",
wantErr: errShortInternal,
},
{
desc: "short internal",
t: mkChain(Nop, 11, rleEncode{}, 3, Nop),
src: "abbcccddddddddddeeeeeeeeeeeg",
dstSize: 3,
srcSize: 100,
wantStr: "1a2b3c10d",
wantErr: errShortInternal,
},
}
}
func doTransform(tc testCase) (res string, iter int, err error) {
reset(tc.t)
dst := make([]byte, tc.dstSize)
out, in := make([]byte, 0, 2*len(tc.src)), []byte(tc.src)
for {
iter++
src, atEOF := in, true
if len(src) > tc.srcSize {
src, atEOF = src[:tc.srcSize], false
}
nDst, nSrc, err := tc.t.Transform(dst, src, atEOF)
out = append(out, dst[:nDst]...)
in = in[nSrc:]
switch {
case err == nil && len(in) != 0:
case err == ErrShortSrc && nSrc > 0:
case err == ErrShortDst && nDst > 0:
default:
return string(out), iter, err
}
}
}
func TestChain(t *testing.T) {
if c, ok := Chain().(nop); !ok {
t.Errorf("empty chain: %v; want Nop", c)
}
// Test Chain for a single Transformer.
for _, tc := range testCases {
tc.t = Chain(tc.t)
str, _, err := doTransform(tc)
if str != tc.wantStr || err != tc.wantErr {
t.Errorf("%s:\ngot %q, %v\nwant %q, %v", tc, str, err, tc.wantStr, tc.wantErr)
}
}
tests := chainTests()
sizes := []int{1, 2, 3, 4, 5, 7, 10, 100, 1000}
addTest := func(tc testCase, t *chain) {
if t.link[0].t != tc.t && tc.wantErr == ErrShortSrc {
tc.wantErr = errShortInternal
}
if t.link[len(t.link)-2].t != tc.t && tc.wantErr == ErrShortDst {
tc.wantErr = errShortInternal
}
tc.t = t
tests = append(tests, tc)
}
for _, tc := range testCases {
for _, sz := range sizes {
tt := tc
tt.dstSize = sz
addTest(tt, mkChain(tc.t, tc.dstSize, Nop))
addTest(tt, mkChain(tc.t, tc.dstSize, Nop, 2, Nop))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop))
if sz >= tc.dstSize && (tc.wantErr != ErrShortDst || sz == tc.dstSize) {
addTest(tt, mkChain(Nop, tc.srcSize, tc.t))
addTest(tt, mkChain(Nop, 100, Nop, tc.srcSize, tc.t))
}
}
}
for _, tc := range testCases {
tt := tc
tt.dstSize = 1
tt.wantStr = ""
addTest(tt, mkChain(tc.t, tc.dstSize, Discard))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Discard))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, tc.dstSize, Discard))
}
for _, tc := range testCases {
tt := tc
tt.dstSize = 100
tt.wantStr = strings.Replace(tc.src, "0f", "", -1)
// Chain encoders and decoders.
if _, ok := tc.t.(rleEncode); ok && tc.wantErr == nil {
addTest(tt, mkChain(tc.t, tc.dstSize, Nop, 1000, rleDecode{}))
addTest(tt, mkChain(tc.t, tc.dstSize, Nop, tc.dstSize, rleDecode{}))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, 100, rleDecode{}))
// decoding needs larger destinations
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, rleDecode{}, 100, Nop))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, 100, rleDecode{}, 100, Nop))
} else if _, ok := tc.t.(rleDecode); ok && tc.wantErr == nil {
// The internal buffer size may need to be the sum of the maximum segment
// size of the two encoders!
addTest(tt, mkChain(tc.t, 2*tc.dstSize, rleEncode{}))
addTest(tt, mkChain(tc.t, tc.dstSize, Nop, 101, rleEncode{}))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, 100, rleEncode{}))
addTest(tt, mkChain(Nop, tc.srcSize, tc.t, tc.dstSize, Nop, 200, rleEncode{}, 100, Nop))
}
}
for _, tc := range tests {
str, iter, err := doTransform(tc)
mi := tc.wantIter != 0 && tc.wantIter != iter
if str != tc.wantStr || err != tc.wantErr || mi {
t.Errorf("%s:\ngot iter:%d, %q, %v\nwant iter:%d, %q, %v", tc, iter, str, err, tc.wantIter, tc.wantStr, tc.wantErr)
}
break
}
}
func TestRemoveFunc(t *testing.T) {
filter := RemoveFunc(func(r rune) bool {
return strings.IndexRune("ab\u0300\u1234,", r) != -1
})
tests := []testCase{
{
src: ",",
wantStr: "",
},
{
src: "c",
wantStr: "c",
},
{
src: "\u2345",
wantStr: "\u2345",
},
{
src: "tschüß",
wantStr: "tschüß",
},
{
src: ",до,свидания,",
wantStr: "досвидания",
},
{
src: "a\xbd\xb2=\xbc ⌘",
wantStr: "\uFFFD\uFFFD=\uFFFD ⌘",
},
{
// If we didn't replace illegal bytes with RuneError, the result
// would be \u0300 or the code would need to be more complex.
src: "\xcc\u0300\x80",
wantStr: "\uFFFD\uFFFD",
},
{
src: "\xcc\u0300\x80",
dstSize: 3,
wantStr: "\uFFFD\uFFFD",
wantIter: 2,
},
{
src: "\u2345",
dstSize: 2,
wantStr: "",
wantErr: ErrShortDst,
},
{
src: "\xcc",
dstSize: 2,
wantStr: "",
wantErr: ErrShortDst,
},
{
src: "\u0300",
dstSize: 2,
srcSize: 1,
wantStr: "",
wantErr: ErrShortSrc,
},
{
t: RemoveFunc(func(r rune) bool {
return r == utf8.RuneError
}),
src: "\xcc\u0300\x80",
wantStr: "\u0300",
},
}
for _, tc := range tests {
tc.desc = tc.src
if tc.t == nil {
tc.t = filter
}
if tc.dstSize == 0 {
tc.dstSize = 100
}
if tc.srcSize == 0 {
tc.srcSize = 100
}
str, iter, err := doTransform(tc)
mi := tc.wantIter != 0 && tc.wantIter != iter
if str != tc.wantStr || err != tc.wantErr || mi {
t.Errorf("%+q:\ngot iter:%d, %+q, %v\nwant iter:%d, %+q, %v", tc.src, iter, str, err, tc.wantIter, tc.wantStr, tc.wantErr)
}
tc.src = str
idem, _, _ := doTransform(tc)
if str != idem {
t.Errorf("%+q: found %+q; want %+q", tc.src, idem, str)
}
}
}
func TestBytes(t *testing.T) {
for _, tt := range append(testCases, chainTests()...) {
if tt.desc == "allowStutter = true" {
// We don't have control over the buffer size, so we eliminate tests
// that depend on a specific buffer size being set.
continue
}
got := Bytes(tt.t, []byte(tt.src))
if tt.wantErr != nil {
if tt.wantErr != ErrShortDst && tt.wantErr != ErrShortSrc {
// Bytes should return nil for non-recoverable errors.
if g, w := (got == nil), (tt.wantErr != nil); g != w {
t.Errorf("%s:error: got %v; want %v", tt.desc, g, w)
}
}
// The output strings in the tests that expect an error will
// almost certainly not be the same as the result of Bytes.
continue
}
if string(got) != tt.wantStr {
t.Errorf("%s:string: got %q; want %q", tt.desc, got, tt.wantStr)
}
}
}

View File

@@ -0,0 +1,30 @@
# Copyright 2011 The Go Authors. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.
maketables: maketables.go triegen.go
go build $^
maketesttables: maketesttables.go triegen.go
go build $^
normregtest: normregtest.go
go build $^
tables: maketables
./maketables > tables.go
gofmt -w tables.go
trietesttables: maketesttables
./maketesttables > triedata_test.go
gofmt -w triedata_test.go
# Downloads from www.unicode.org, so not part
# of standard test scripts.
test: testtables regtest
testtables: maketables
./maketables -test > data_test.go && go test -tags=test
regtest: normregtest
./normregtest

View File

@@ -0,0 +1,514 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import "unicode/utf8"
const (
maxNonStarters = 30
// The maximum number of characters needed for a buffer is
// maxNonStarters + 1 for the starter + 1 for the GCJ
maxBufferSize = maxNonStarters + 2
maxNFCExpansion = 3 // NFC(0x1D160)
maxNFKCExpansion = 18 // NFKC(0xFDFA)
maxByteBufferSize = utf8.UTFMax * maxBufferSize // 128
)
// ssState is used for reporting the segment state after inserting a rune.
// It is returned by streamSafe.next.
type ssState int
const (
// Indicates a rune was successfully added to the segment.
ssSuccess ssState = iota
// Indicates a rune starts a new segment and should not be added.
ssStarter
// Indicates a rune caused a segment overflow and a CGJ should be inserted.
ssOverflow
)
// streamSafe implements the policy of when a CGJ should be inserted.
type streamSafe uint8
// mkStreamSafe is a shorthand for declaring a streamSafe var and calling
// first on it.
func mkStreamSafe(p Properties) streamSafe {
return streamSafe(p.nTrailingNonStarters())
}
// first inserts the first rune of a segment.
func (ss *streamSafe) first(p Properties) {
if *ss != 0 {
panic("!= 0")
}
*ss = streamSafe(p.nTrailingNonStarters())
}
// insert returns a ssState value to indicate whether a rune represented by p
// can be inserted.
func (ss *streamSafe) next(p Properties) ssState {
if *ss > maxNonStarters {
panic("streamSafe was not reset")
}
n := p.nLeadingNonStarters()
if *ss += streamSafe(n); *ss > maxNonStarters {
*ss = 0
return ssOverflow
}
// The Stream-Safe Text Processing prescribes that the counting can stop
// as soon as a starter is encountered. However, there are some starters,
// like Jamo V and T, that can combine with other runes, leaving their
// successive non-starters appended to the previous, possibly causing an
// overflow. We will therefore consider any rune with a non-zero nLead to
// be a non-starter. Note that it always hold that if nLead > 0 then
// nLead == nTrail.
if n == 0 {
*ss = 0
return ssStarter
}
return ssSuccess
}
// backwards is used for checking for overflow and segment starts
// when traversing a string backwards. Users do not need to call first
// for the first rune. The state of the streamSafe retains the count of
// the non-starters loaded.
func (ss *streamSafe) backwards(p Properties) ssState {
if *ss > maxNonStarters {
panic("streamSafe was not reset")
}
c := *ss + streamSafe(p.nTrailingNonStarters())
if c > maxNonStarters {
return ssOverflow
}
*ss = c
if p.nLeadingNonStarters() == 0 {
return ssStarter
}
return ssSuccess
}
func (ss streamSafe) isMax() bool {
return ss == maxNonStarters
}
// GraphemeJoiner is inserted after maxNonStarters non-starter runes.
const GraphemeJoiner = "\u034F"
// reorderBuffer is used to normalize a single segment. Characters inserted with
// insert are decomposed and reordered based on CCC. The compose method can
// be used to recombine characters. Note that the byte buffer does not hold
// the UTF-8 characters in order. Only the rune array is maintained in sorted
// order. flush writes the resulting segment to a byte array.
type reorderBuffer struct {
rune [maxBufferSize]Properties // Per character info.
byte [maxByteBufferSize]byte // UTF-8 buffer. Referenced by runeInfo.pos.
nbyte uint8 // Number or bytes.
ss streamSafe // For limiting length of non-starter sequence.
nrune int // Number of runeInfos.
f formInfo
src input
nsrc int
tmpBytes input
out []byte
flushF func(*reorderBuffer) bool
}
func (rb *reorderBuffer) init(f Form, src []byte) {
rb.f = *formTable[f]
rb.src.setBytes(src)
rb.nsrc = len(src)
rb.ss = 0
}
func (rb *reorderBuffer) initString(f Form, src string) {
rb.f = *formTable[f]
rb.src.setString(src)
rb.nsrc = len(src)
rb.ss = 0
}
func (rb *reorderBuffer) setFlusher(out []byte, f func(*reorderBuffer) bool) {
rb.out = out
rb.flushF = f
}
// reset discards all characters from the buffer.
func (rb *reorderBuffer) reset() {
rb.nrune = 0
rb.nbyte = 0
rb.ss = 0
}
func (rb *reorderBuffer) doFlush() bool {
if rb.f.composing {
rb.compose()
}
res := rb.flushF(rb)
rb.reset()
return res
}
// appendFlush appends the normalized segment to rb.out.
func appendFlush(rb *reorderBuffer) bool {
for i := 0; i < rb.nrune; i++ {
start := rb.rune[i].pos
end := start + rb.rune[i].size
rb.out = append(rb.out, rb.byte[start:end]...)
}
return true
}
// flush appends the normalized segment to out and resets rb.
func (rb *reorderBuffer) flush(out []byte) []byte {
for i := 0; i < rb.nrune; i++ {
start := rb.rune[i].pos
end := start + rb.rune[i].size
out = append(out, rb.byte[start:end]...)
}
rb.reset()
return out
}
// flushCopy copies the normalized segment to buf and resets rb.
// It returns the number of bytes written to buf.
func (rb *reorderBuffer) flushCopy(buf []byte) int {
p := 0
for i := 0; i < rb.nrune; i++ {
runep := rb.rune[i]
p += copy(buf[p:], rb.byte[runep.pos:runep.pos+runep.size])
}
rb.reset()
return p
}
// insertOrdered inserts a rune in the buffer, ordered by Canonical Combining Class.
// It returns false if the buffer is not large enough to hold the rune.
// It is used internally by insert and insertString only.
func (rb *reorderBuffer) insertOrdered(info Properties) {
n := rb.nrune
b := rb.rune[:]
cc := info.ccc
if cc > 0 {
// Find insertion position + move elements to make room.
for ; n > 0; n-- {
if b[n-1].ccc <= cc {
break
}
b[n] = b[n-1]
}
}
rb.nrune += 1
pos := uint8(rb.nbyte)
rb.nbyte += utf8.UTFMax
info.pos = pos
b[n] = info
}
// insertErr is an error code returned by insert. Using this type instead
// of error improves performance up to 20% for many of the benchmarks.
type insertErr int
const (
iSuccess insertErr = -iota
iShortDst
iShortSrc
)
// insertFlush inserts the given rune in the buffer ordered by CCC.
// If a decomposition with multiple segments are encountered, they leading
// ones are flushed.
// It returns a non-zero error code if the rune was not inserted.
func (rb *reorderBuffer) insertFlush(src input, i int, info Properties) insertErr {
if rune := src.hangul(i); rune != 0 {
rb.decomposeHangul(rune)
return iSuccess
}
if info.hasDecomposition() {
return rb.insertDecomposed(info.Decomposition())
}
rb.insertSingle(src, i, info)
return iSuccess
}
// insertUnsafe inserts the given rune in the buffer ordered by CCC.
// It is assumed there is sufficient space to hold the runes. It is the
// responsibility of the caller to ensure this. This can be done by checking
// the state returned by the streamSafe type.
func (rb *reorderBuffer) insertUnsafe(src input, i int, info Properties) {
if rune := src.hangul(i); rune != 0 {
rb.decomposeHangul(rune)
}
if info.hasDecomposition() {
// TODO: inline.
rb.insertDecomposed(info.Decomposition())
} else {
rb.insertSingle(src, i, info)
}
}
// insertDecomposed inserts an entry in to the reorderBuffer for each rune
// in dcomp. dcomp must be a sequence of decomposed UTF-8-encoded runes.
// It flushes the buffer on each new segment start.
func (rb *reorderBuffer) insertDecomposed(dcomp []byte) insertErr {
rb.tmpBytes.setBytes(dcomp)
for i := 0; i < len(dcomp); {
info := rb.f.info(rb.tmpBytes, i)
if info.BoundaryBefore() && rb.nrune > 0 && !rb.doFlush() {
return iShortDst
}
i += copy(rb.byte[rb.nbyte:], dcomp[i:i+int(info.size)])
rb.insertOrdered(info)
}
return iSuccess
}
// insertSingle inserts an entry in the reorderBuffer for the rune at
// position i. info is the runeInfo for the rune at position i.
func (rb *reorderBuffer) insertSingle(src input, i int, info Properties) {
src.copySlice(rb.byte[rb.nbyte:], i, i+int(info.size))
rb.insertOrdered(info)
}
// insertCGJ inserts a Combining Grapheme Joiner (0x034f) into rb.
func (rb *reorderBuffer) insertCGJ() {
rb.insertSingle(input{str: GraphemeJoiner}, 0, Properties{size: uint8(len(GraphemeJoiner))})
}
// appendRune inserts a rune at the end of the buffer. It is used for Hangul.
func (rb *reorderBuffer) appendRune(r rune) {
bn := rb.nbyte
sz := utf8.EncodeRune(rb.byte[bn:], rune(r))
rb.nbyte += utf8.UTFMax
rb.rune[rb.nrune] = Properties{pos: bn, size: uint8(sz)}
rb.nrune++
}
// assignRune sets a rune at position pos. It is used for Hangul and recomposition.
func (rb *reorderBuffer) assignRune(pos int, r rune) {
bn := rb.rune[pos].pos
sz := utf8.EncodeRune(rb.byte[bn:], rune(r))
rb.rune[pos] = Properties{pos: bn, size: uint8(sz)}
}
// runeAt returns the rune at position n. It is used for Hangul and recomposition.
func (rb *reorderBuffer) runeAt(n int) rune {
inf := rb.rune[n]
r, _ := utf8.DecodeRune(rb.byte[inf.pos : inf.pos+inf.size])
return r
}
// bytesAt returns the UTF-8 encoding of the rune at position n.
// It is used for Hangul and recomposition.
func (rb *reorderBuffer) bytesAt(n int) []byte {
inf := rb.rune[n]
return rb.byte[inf.pos : int(inf.pos)+int(inf.size)]
}
// For Hangul we combine algorithmically, instead of using tables.
const (
hangulBase = 0xAC00 // UTF-8(hangulBase) -> EA B0 80
hangulBase0 = 0xEA
hangulBase1 = 0xB0
hangulBase2 = 0x80
hangulEnd = hangulBase + jamoLVTCount // UTF-8(0xD7A4) -> ED 9E A4
hangulEnd0 = 0xED
hangulEnd1 = 0x9E
hangulEnd2 = 0xA4
jamoLBase = 0x1100 // UTF-8(jamoLBase) -> E1 84 00
jamoLBase0 = 0xE1
jamoLBase1 = 0x84
jamoLEnd = 0x1113
jamoVBase = 0x1161
jamoVEnd = 0x1176
jamoTBase = 0x11A7
jamoTEnd = 0x11C3
jamoTCount = 28
jamoVCount = 21
jamoVTCount = 21 * 28
jamoLVTCount = 19 * 21 * 28
)
const hangulUTF8Size = 3
func isHangul(b []byte) bool {
if len(b) < hangulUTF8Size {
return false
}
b0 := b[0]
if b0 < hangulBase0 {
return false
}
b1 := b[1]
switch {
case b0 == hangulBase0:
return b1 >= hangulBase1
case b0 < hangulEnd0:
return true
case b0 > hangulEnd0:
return false
case b1 < hangulEnd1:
return true
}
return b1 == hangulEnd1 && b[2] < hangulEnd2
}
func isHangulString(b string) bool {
if len(b) < hangulUTF8Size {
return false
}
b0 := b[0]
if b0 < hangulBase0 {
return false
}
b1 := b[1]
switch {
case b0 == hangulBase0:
return b1 >= hangulBase1
case b0 < hangulEnd0:
return true
case b0 > hangulEnd0:
return false
case b1 < hangulEnd1:
return true
}
return b1 == hangulEnd1 && b[2] < hangulEnd2
}
// Caller must ensure len(b) >= 2.
func isJamoVT(b []byte) bool {
// True if (rune & 0xff00) == jamoLBase
return b[0] == jamoLBase0 && (b[1]&0xFC) == jamoLBase1
}
func isHangulWithoutJamoT(b []byte) bool {
c, _ := utf8.DecodeRune(b)
c -= hangulBase
return c < jamoLVTCount && c%jamoTCount == 0
}
// decomposeHangul writes the decomposed Hangul to buf and returns the number
// of bytes written. len(buf) should be at least 9.
func decomposeHangul(buf []byte, r rune) int {
const JamoUTF8Len = 3
r -= hangulBase
x := r % jamoTCount
r /= jamoTCount
utf8.EncodeRune(buf, jamoLBase+r/jamoVCount)
utf8.EncodeRune(buf[JamoUTF8Len:], jamoVBase+r%jamoVCount)
if x != 0 {
utf8.EncodeRune(buf[2*JamoUTF8Len:], jamoTBase+x)
return 3 * JamoUTF8Len
}
return 2 * JamoUTF8Len
}
// decomposeHangul algorithmically decomposes a Hangul rune into
// its Jamo components.
// See http://unicode.org/reports/tr15/#Hangul for details on decomposing Hangul.
func (rb *reorderBuffer) decomposeHangul(r rune) {
r -= hangulBase
x := r % jamoTCount
r /= jamoTCount
rb.appendRune(jamoLBase + r/jamoVCount)
rb.appendRune(jamoVBase + r%jamoVCount)
if x != 0 {
rb.appendRune(jamoTBase + x)
}
}
// combineHangul algorithmically combines Jamo character components into Hangul.
// See http://unicode.org/reports/tr15/#Hangul for details on combining Hangul.
func (rb *reorderBuffer) combineHangul(s, i, k int) {
b := rb.rune[:]
bn := rb.nrune
for ; i < bn; i++ {
cccB := b[k-1].ccc
cccC := b[i].ccc
if cccB == 0 {
s = k - 1
}
if s != k-1 && cccB >= cccC {
// b[i] is blocked by greater-equal cccX below it
b[k] = b[i]
k++
} else {
l := rb.runeAt(s) // also used to compare to hangulBase
v := rb.runeAt(i) // also used to compare to jamoT
switch {
case jamoLBase <= l && l < jamoLEnd &&
jamoVBase <= v && v < jamoVEnd:
// 11xx plus 116x to LV
rb.assignRune(s, hangulBase+
(l-jamoLBase)*jamoVTCount+(v-jamoVBase)*jamoTCount)
case hangulBase <= l && l < hangulEnd &&
jamoTBase < v && v < jamoTEnd &&
((l-hangulBase)%jamoTCount) == 0:
// ACxx plus 11Ax to LVT
rb.assignRune(s, l+v-jamoTBase)
default:
b[k] = b[i]
k++
}
}
}
rb.nrune = k
}
// compose recombines the runes in the buffer.
// It should only be used to recompose a single segment, as it will not
// handle alternations between Hangul and non-Hangul characters correctly.
func (rb *reorderBuffer) compose() {
// UAX #15, section X5 , including Corrigendum #5
// "In any character sequence beginning with starter S, a character C is
// blocked from S if and only if there is some character B between S
// and C, and either B is a starter or it has the same or higher
// combining class as C."
bn := rb.nrune
if bn == 0 {
return
}
k := 1
b := rb.rune[:]
for s, i := 0, 1; i < bn; i++ {
if isJamoVT(rb.bytesAt(i)) {
// Redo from start in Hangul mode. Necessary to support
// U+320E..U+321E in NFKC mode.
rb.combineHangul(s, i, k)
return
}
ii := b[i]
// We can only use combineForward as a filter if we later
// get the info for the combined character. This is more
// expensive than using the filter. Using combinesBackward()
// is safe.
if ii.combinesBackward() {
cccB := b[k-1].ccc
cccC := ii.ccc
blocked := false // b[i] blocked by starter or greater or equal CCC?
if cccB == 0 {
s = k - 1
} else {
blocked = s != k-1 && cccB >= cccC
}
if !blocked {
combined := combine(rb.runeAt(s), rb.runeAt(i))
if combined != 0 {
rb.assignRune(s, combined)
continue
}
}
}
b[k] = b[i]
k++
}
rb.nrune = k
}

View File

@@ -0,0 +1,130 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import "testing"
// TestCase is used for most tests.
type TestCase struct {
in []rune
out []rune
}
func runTests(t *testing.T, name string, fm Form, tests []TestCase) {
rb := reorderBuffer{}
rb.init(fm, nil)
for i, test := range tests {
rb.setFlusher(nil, appendFlush)
for j, rune := range test.in {
b := []byte(string(rune))
src := inputBytes(b)
info := rb.f.info(src, 0)
if j == 0 {
rb.ss.first(info)
} else {
rb.ss.next(info)
}
if rb.insertFlush(src, 0, info) < 0 {
t.Errorf("%s:%d: insert failed for rune %d", name, i, j)
}
}
rb.doFlush()
was := string(rb.out)
want := string(test.out)
if len(was) != len(want) {
t.Errorf("%s:%d: length = %d; want %d", name, i, len(was), len(want))
}
if was != want {
k, pfx := pidx(was, want)
t.Errorf("%s:%d: \nwas %s%+q; \nwant %s%+q", name, i, pfx, was[k:], pfx, want[k:])
}
}
}
func TestFlush(t *testing.T) {
const (
hello = "Hello "
world = "world!"
)
buf := make([]byte, maxByteBufferSize)
p := copy(buf, hello)
out := buf[p:]
rb := reorderBuffer{}
rb.initString(NFC, world)
if i := rb.flushCopy(out); i != 0 {
t.Errorf("wrote bytes on flush of empty buffer. (len(out) = %d)", i)
}
for i := range world {
// No need to set streamSafe values for this test.
rb.insertFlush(rb.src, i, rb.f.info(rb.src, i))
n := rb.flushCopy(out)
out = out[n:]
p += n
}
was := buf[:p]
want := hello + world
if string(was) != want {
t.Errorf(`output after flush was "%s"; want "%s"`, string(was), want)
}
if rb.nrune != 0 {
t.Errorf("non-null size of info buffer (rb.nrune == %d)", rb.nrune)
}
if rb.nbyte != 0 {
t.Errorf("non-null size of byte buffer (rb.nbyte == %d)", rb.nbyte)
}
}
var insertTests = []TestCase{
{[]rune{'a'}, []rune{'a'}},
{[]rune{0x300}, []rune{0x300}},
{[]rune{0x300, 0x316}, []rune{0x316, 0x300}}, // CCC(0x300)==230; CCC(0x316)==220
{[]rune{0x316, 0x300}, []rune{0x316, 0x300}},
{[]rune{0x41, 0x316, 0x300}, []rune{0x41, 0x316, 0x300}},
{[]rune{0x41, 0x300, 0x316}, []rune{0x41, 0x316, 0x300}},
{[]rune{0x300, 0x316, 0x41}, []rune{0x316, 0x300, 0x41}},
{[]rune{0x41, 0x300, 0x40, 0x316}, []rune{0x41, 0x300, 0x40, 0x316}},
}
func TestInsert(t *testing.T) {
runTests(t, "TestInsert", NFD, insertTests)
}
var decompositionNFDTest = []TestCase{
{[]rune{0xC0}, []rune{0x41, 0x300}},
{[]rune{0xAC00}, []rune{0x1100, 0x1161}},
{[]rune{0x01C4}, []rune{0x01C4}},
{[]rune{0x320E}, []rune{0x320E}},
{[]rune("음ẻ과"), []rune{0x110B, 0x1173, 0x11B7, 0x65, 0x309, 0x1100, 0x116A}},
}
var decompositionNFKDTest = []TestCase{
{[]rune{0xC0}, []rune{0x41, 0x300}},
{[]rune{0xAC00}, []rune{0x1100, 0x1161}},
{[]rune{0x01C4}, []rune{0x44, 0x5A, 0x030C}},
{[]rune{0x320E}, []rune{0x28, 0x1100, 0x1161, 0x29}},
}
func TestDecomposition(t *testing.T) {
runTests(t, "TestDecompositionNFD", NFD, decompositionNFDTest)
runTests(t, "TestDecompositionNFKD", NFKD, decompositionNFKDTest)
}
var compositionTest = []TestCase{
{[]rune{0x41, 0x300}, []rune{0xC0}},
{[]rune{0x41, 0x316}, []rune{0x41, 0x316}},
{[]rune{0x41, 0x300, 0x35D}, []rune{0xC0, 0x35D}},
{[]rune{0x41, 0x316, 0x300}, []rune{0xC0, 0x316}},
// blocking starter
{[]rune{0x41, 0x316, 0x40, 0x300}, []rune{0x41, 0x316, 0x40, 0x300}},
{[]rune{0x1100, 0x1161}, []rune{0xAC00}},
// parenthesized Hangul, alternate between ASCII and Hangul.
{[]rune{0x28, 0x1100, 0x1161, 0x29}, []rune{0x28, 0xAC00, 0x29}},
}
func TestComposition(t *testing.T) {
runTests(t, "TestComposition", NFC, compositionTest)
}

View File

@@ -0,0 +1,82 @@
// Copyright 2012 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm_test
import (
"bytes"
"fmt"
"unicode/utf8"
"code.google.com/p/go.text/unicode/norm"
)
// EqualSimple uses a norm.Iter to compare two non-normalized
// strings for equivalence.
func EqualSimple(a, b string) bool {
var ia, ib norm.Iter
ia.InitString(norm.NFKD, a)
ib.InitString(norm.NFKD, b)
for !ia.Done() && !ib.Done() {
if !bytes.Equal(ia.Next(), ib.Next()) {
return false
}
}
return ia.Done() && ib.Done()
}
// FindPrefix finds the longest common prefix of ASCII characters
// of a and b.
func FindPrefix(a, b string) int {
i := 0
for ; i < len(a) && i < len(b) && a[i] < utf8.RuneSelf && a[i] == b[i]; i++ {
}
return i
}
// EqualOpt is like EqualSimple, but optimizes the special
// case for ASCII characters.
func EqualOpt(a, b string) bool {
n := FindPrefix(a, b)
a, b = a[n:], b[n:]
var ia, ib norm.Iter
ia.InitString(norm.NFKD, a)
ib.InitString(norm.NFKD, b)
for !ia.Done() && !ib.Done() {
if !bytes.Equal(ia.Next(), ib.Next()) {
return false
}
if n := int64(FindPrefix(a[ia.Pos():], b[ib.Pos():])); n != 0 {
ia.Seek(n, 1)
ib.Seek(n, 1)
}
}
return ia.Done() && ib.Done()
}
var compareTests = []struct{ a, b string }{
{"aaa", "aaa"},
{"aaa", "aab"},
{"a\u0300a", "\u00E0a"},
{"a\u0300\u0320b", "a\u0320\u0300b"},
{"\u1E0A\u0323", "\x44\u0323\u0307"},
// A character that decomposes into multiple segments
// spans several iterations.
{"\u3304", "\u30A4\u30CB\u30F3\u30AF\u3099"},
}
func ExampleIter() {
for i, t := range compareTests {
r0 := EqualSimple(t.a, t.b)
r1 := EqualOpt(t.a, t.b)
fmt.Printf("%d: %v %v\n", i, r0, r1)
}
// Output:
// 0: true true
// 1: false false
// 2: true true
// 3: true true
// 4: true true
// 5: true true
}

View File

@@ -0,0 +1,256 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
// This file contains Form-specific logic and wrappers for data in tables.go.
// Rune info is stored in a separate trie per composing form. A composing form
// and its corresponding decomposing form share the same trie. Each trie maps
// a rune to a uint16. The values take two forms. For v >= 0x8000:
// bits
// 15: 1 (inverse of NFD_QD bit of qcInfo)
// 13..7: qcInfo (see below). isYesD is always true (no decompostion).
// 6..0: ccc (compressed CCC value).
// For v < 0x8000, the respective rune has a decomposition and v is an index
// into a byte array of UTF-8 decomposition sequences and additional info and
// has the form:
// <header> <decomp_byte>* [<tccc> [<lccc>]]
// The header contains the number of bytes in the decomposition (excluding this
// length byte). The two most significant bits of this length byte correspond
// to bit 5 and 4 of qcInfo (see below). The byte sequence itself starts at v+1.
// The byte sequence is followed by a trailing and leading CCC if the values
// for these are not zero. The value of v determines which ccc are appended
// to the sequences. For v < firstCCC, there are none, for v >= firstCCC,
// the sequence is followed by a trailing ccc, and for v >= firstLeadingCC
// there is an additional leading ccc. The value of tccc itself is the
// trailing CCC shifted left 2 bits. The two least-significant bits of tccc
// are the number of trailing non-starters.
const (
qcInfoMask = 0x3F // to clear all but the relevant bits in a qcInfo
headerLenMask = 0x3F // extract the length value from the header byte
headerFlagsMask = 0xC0 // extract the qcInfo bits from the header byte
)
// Properties provides access to normalization properties of a rune.
type Properties struct {
pos uint8 // start position in reorderBuffer; used in composition.go
size uint8 // length of UTF-8 encoding of this rune
ccc uint8 // leading canonical combining class (ccc if not decomposition)
tccc uint8 // trailing canonical combining class (ccc if not decomposition)
nLead uint8 // number of leading non-starters.
flags qcInfo // quick check flags
index uint16
}
// functions dispatchable per form
type lookupFunc func(b input, i int) Properties
// formInfo holds Form-specific functions and tables.
type formInfo struct {
form Form
composing, compatibility bool // form type
info lookupFunc
nextMain iterFunc
}
var formTable []*formInfo
func init() {
formTable = make([]*formInfo, 4)
for i := range formTable {
f := &formInfo{}
formTable[i] = f
f.form = Form(i)
if Form(i) == NFKD || Form(i) == NFKC {
f.compatibility = true
f.info = lookupInfoNFKC
} else {
f.info = lookupInfoNFC
}
f.nextMain = nextDecomposed
if Form(i) == NFC || Form(i) == NFKC {
f.nextMain = nextComposed
f.composing = true
}
}
}
// We do not distinguish between boundaries for NFC, NFD, etc. to avoid
// unexpected behavior for the user. For example, in NFD, there is a boundary
// after 'a'. However, 'a' might combine with modifiers, so from the application's
// perspective it is not a good boundary. We will therefore always use the
// boundaries for the combining variants.
// BoundaryBefore returns true if this rune starts a new segment and
// cannot combine with any rune on the left.
func (p Properties) BoundaryBefore() bool {
if p.ccc == 0 && !p.combinesBackward() {
return true
}
// We assume that the CCC of the first character in a decomposition
// is always non-zero if different from info.ccc and that we can return
// false at this point. This is verified by maketables.
return false
}
// BoundaryAfter returns true if runes cannot combine with or otherwise
// interact with this or previous runes.
func (p Properties) BoundaryAfter() bool {
// TODO: loosen these conditions.
return p.isInert()
}
// We pack quick check data in 4 bits:
// 5: Combines forward (0 == false, 1 == true)
// 4..3: NFC_QC Yes(00), No (10), or Maybe (11)
// 2: NFD_QC Yes (0) or No (1). No also means there is a decomposition.
// 1..0: Number of trailing non-starters.
//
// When all 4 bits are zero, the character is inert, meaning it is never
// influenced by normalization.
type qcInfo uint8
func (p Properties) isYesC() bool { return p.flags&0x10 == 0 }
func (p Properties) isYesD() bool { return p.flags&0x4 == 0 }
func (p Properties) combinesForward() bool { return p.flags&0x20 != 0 }
func (p Properties) combinesBackward() bool { return p.flags&0x8 != 0 } // == isMaybe
func (p Properties) hasDecomposition() bool { return p.flags&0x4 != 0 } // == isNoD
func (p Properties) isInert() bool {
return p.flags&qcInfoMask == 0 && p.ccc == 0
}
func (p Properties) multiSegment() bool {
return p.index >= firstMulti && p.index < endMulti
}
func (p Properties) nLeadingNonStarters() uint8 {
return p.nLead
}
func (p Properties) nTrailingNonStarters() uint8 {
return uint8(p.flags & 0x03)
}
// Decomposition returns the decomposition for the underlying rune
// or nil if there is none.
func (p Properties) Decomposition() []byte {
// TODO: create the decomposition for Hangul?
if p.index == 0 {
return nil
}
i := p.index
n := decomps[i] & headerLenMask
i++
return decomps[i : i+uint16(n)]
}
// Size returns the length of UTF-8 encoding of the rune.
func (p Properties) Size() int {
return int(p.size)
}
// CCC returns the canonical combining class of the underlying rune.
func (p Properties) CCC() uint8 {
if p.index >= firstCCCZeroExcept {
return 0
}
return ccc[p.ccc]
}
// LeadCCC returns the CCC of the first rune in the decomposition.
// If there is no decomposition, LeadCCC equals CCC.
func (p Properties) LeadCCC() uint8 {
return ccc[p.ccc]
}
// TrailCCC returns the CCC of the last rune in the decomposition.
// If there is no decomposition, TrailCCC equals CCC.
func (p Properties) TrailCCC() uint8 {
return ccc[p.tccc]
}
// Recomposition
// We use 32-bit keys instead of 64-bit for the two codepoint keys.
// This clips off the bits of three entries, but we know this will not
// result in a collision. In the unlikely event that changes to
// UnicodeData.txt introduce collisions, the compiler will catch it.
// Note that the recomposition map for NFC and NFKC are identical.
// combine returns the combined rune or 0 if it doesn't exist.
func combine(a, b rune) rune {
key := uint32(uint16(a))<<16 + uint32(uint16(b))
return recompMap[key]
}
func lookupInfoNFC(b input, i int) Properties {
v, sz := b.charinfoNFC(i)
return compInfo(v, sz)
}
func lookupInfoNFKC(b input, i int) Properties {
v, sz := b.charinfoNFKC(i)
return compInfo(v, sz)
}
// Properties returns properties for the first rune in s.
func (f Form) Properties(s []byte) Properties {
if f == NFC || f == NFD {
return compInfo(nfcTrie.lookup(s))
}
return compInfo(nfkcTrie.lookup(s))
}
// PropertiesString returns properties for the first rune in s.
func (f Form) PropertiesString(s string) Properties {
if f == NFC || f == NFD {
return compInfo(nfcTrie.lookupString(s))
}
return compInfo(nfkcTrie.lookupString(s))
}
// compInfo converts the information contained in v and sz
// to a Properties. See the comment at the top of the file
// for more information on the format.
func compInfo(v uint16, sz int) Properties {
if v == 0 {
return Properties{size: uint8(sz)}
} else if v >= 0x8000 {
p := Properties{
size: uint8(sz),
ccc: uint8(v),
tccc: uint8(v),
flags: qcInfo(v >> 8),
}
if p.ccc > 0 || p.combinesBackward() {
p.nLead = uint8(p.flags & 0x3)
}
return p
}
// has decomposition
h := decomps[v]
f := (qcInfo(h&headerFlagsMask) >> 2) | 0x4
p := Properties{size: uint8(sz), flags: f, index: v}
if v >= firstCCC {
v += uint16(h&headerLenMask) + 1
c := decomps[v]
p.tccc = c >> 2
p.flags |= qcInfo(c & 0x3)
if v >= firstLeadingCCC {
p.nLead = c & 0x3
if v >= firstStarterWithNLead {
// We were tricked. Remove the decomposition.
p.flags &= 0x03
p.index = 0
return p
}
p.ccc = decomps[v+1]
}
}
return p
}

View File

@@ -0,0 +1,54 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build test
package norm
import "testing"
func TestProperties(t *testing.T) {
var d runeData
CK := [2]string{"C", "K"}
for k, r := 1, rune(0); r < 0x2ffff; r++ {
if k < len(testData) && r == testData[k].r {
d = testData[k]
k++
}
s := string(r)
for j, p := range []Properties{NFC.PropertiesString(s), NFKC.PropertiesString(s)} {
f := d.f[j]
if p.CCC() != d.ccc {
t.Errorf("%U: ccc(%s): was %d; want %d %X", r, CK[j], p.CCC(), d.ccc, p.index)
}
if p.isYesC() != (f.qc == Yes) {
t.Errorf("%U: YesC(%s): was %v; want %v", r, CK[j], p.isYesC(), f.qc == Yes)
}
if p.combinesBackward() != (f.qc == Maybe) {
t.Errorf("%U: combines backwards(%s): was %v; want %v", r, CK[j], p.combinesBackward(), f.qc == Maybe)
}
if p.nLeadingNonStarters() != d.nLead {
t.Errorf("%U: nLead(%s): was %d; want %d %#v %#v", r, CK[j], p.nLeadingNonStarters(), d.nLead, p, d)
}
if p.nTrailingNonStarters() != d.nTrail {
t.Errorf("%U: nTrail(%s): was %d; want %d %#v %#v", r, CK[j], p.nTrailingNonStarters(), d.nTrail, p, d)
}
if p.combinesForward() != f.combinesForward {
t.Errorf("%U: combines forward(%s): was %v; want %v %#v", r, CK[j], p.combinesForward(), f.combinesForward, p)
}
// Skip Hangul as it is algorithmically computed.
if r >= hangulBase && r < hangulEnd {
continue
}
if p.hasDecomposition() {
if has := f.decomposition != ""; !has {
t.Errorf("%U: hasDecomposition(%s): was %v; want %v", r, CK[j], p.hasDecomposition(), has)
}
if string(p.Decomposition()) != f.decomposition {
t.Errorf("%U: decomp(%s): was %+q; want %+q", r, CK[j], p.Decomposition(), f.decomposition)
}
}
}
}
}

View File

@@ -0,0 +1,105 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import "unicode/utf8"
type input struct {
str string
bytes []byte
}
func inputBytes(str []byte) input {
return input{bytes: str}
}
func inputString(str string) input {
return input{str: str}
}
func (in *input) setBytes(str []byte) {
in.str = ""
in.bytes = str
}
func (in *input) setString(str string) {
in.str = str
in.bytes = nil
}
func (in *input) _byte(p int) byte {
if in.bytes == nil {
return in.str[p]
}
return in.bytes[p]
}
func (in *input) skipASCII(p, max int) int {
if in.bytes == nil {
for ; p < max && in.str[p] < utf8.RuneSelf; p++ {
}
} else {
for ; p < max && in.bytes[p] < utf8.RuneSelf; p++ {
}
}
return p
}
func (in *input) skipContinuationBytes(p int) int {
if in.bytes == nil {
for ; p < len(in.str) && !utf8.RuneStart(in.str[p]); p++ {
}
} else {
for ; p < len(in.bytes) && !utf8.RuneStart(in.bytes[p]); p++ {
}
}
return p
}
func (in *input) appendSlice(buf []byte, b, e int) []byte {
if in.bytes != nil {
return append(buf, in.bytes[b:e]...)
}
for i := b; i < e; i++ {
buf = append(buf, in.str[i])
}
return buf
}
func (in *input) copySlice(buf []byte, b, e int) int {
if in.bytes == nil {
return copy(buf, in.str[b:e])
}
return copy(buf, in.bytes[b:e])
}
func (in *input) charinfoNFC(p int) (uint16, int) {
if in.bytes == nil {
return nfcTrie.lookupString(in.str[p:])
}
return nfcTrie.lookup(in.bytes[p:])
}
func (in *input) charinfoNFKC(p int) (uint16, int) {
if in.bytes == nil {
return nfkcTrie.lookupString(in.str[p:])
}
return nfkcTrie.lookup(in.bytes[p:])
}
func (in *input) hangul(p int) (r rune) {
if in.bytes == nil {
if !isHangulString(in.str[p:]) {
return 0
}
r, _ = utf8.DecodeRuneInString(in.str[p:])
} else {
if !isHangul(in.bytes[p:]) {
return 0
}
r, _ = utf8.DecodeRune(in.bytes[p:])
}
return r
}

View File

@@ -0,0 +1,448 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"fmt"
"unicode/utf8"
)
const MaxSegmentSize = maxByteBufferSize
// An Iter iterates over a string or byte slice, while normalizing it
// to a given Form.
type Iter struct {
rb reorderBuffer
buf [maxByteBufferSize]byte
info Properties // first character saved from previous iteration
next iterFunc // implementation of next depends on form
asciiF iterFunc
p int // current position in input source
multiSeg []byte // remainder of multi-segment decomposition
}
type iterFunc func(*Iter) []byte
// Init initializes i to iterate over src after normalizing it to Form f.
func (i *Iter) Init(f Form, src []byte) {
i.p = 0
if len(src) == 0 {
i.setDone()
i.rb.nsrc = 0
return
}
i.multiSeg = nil
i.rb.init(f, src)
i.next = i.rb.f.nextMain
i.asciiF = nextASCIIBytes
i.info = i.rb.f.info(i.rb.src, i.p)
}
// InitString initializes i to iterate over src after normalizing it to Form f.
func (i *Iter) InitString(f Form, src string) {
i.p = 0
if len(src) == 0 {
i.setDone()
i.rb.nsrc = 0
return
}
i.multiSeg = nil
i.rb.initString(f, src)
i.next = i.rb.f.nextMain
i.asciiF = nextASCIIString
i.info = i.rb.f.info(i.rb.src, i.p)
}
// Seek sets the segment to be returned by the next call to Next to start
// at position p. It is the responsibility of the caller to set p to the
// start of a UTF8 rune.
func (i *Iter) Seek(offset int64, whence int) (int64, error) {
var abs int64
switch whence {
case 0:
abs = offset
case 1:
abs = int64(i.p) + offset
case 2:
abs = int64(i.rb.nsrc) + offset
default:
return 0, fmt.Errorf("norm: invalid whence")
}
if abs < 0 {
return 0, fmt.Errorf("norm: negative position")
}
if int(abs) >= i.rb.nsrc {
i.setDone()
return int64(i.p), nil
}
i.p = int(abs)
i.multiSeg = nil
i.next = i.rb.f.nextMain
i.info = i.rb.f.info(i.rb.src, i.p)
return abs, nil
}
// returnSlice returns a slice of the underlying input type as a byte slice.
// If the underlying is of type []byte, it will simply return a slice.
// If the underlying is of type string, it will copy the slice to the buffer
// and return that.
func (i *Iter) returnSlice(a, b int) []byte {
if i.rb.src.bytes == nil {
return i.buf[:copy(i.buf[:], i.rb.src.str[a:b])]
}
return i.rb.src.bytes[a:b]
}
// Pos returns the byte position at which the next call to Next will commence processing.
func (i *Iter) Pos() int {
return i.p
}
func (i *Iter) setDone() {
i.next = nextDone
i.p = i.rb.nsrc
}
// Done returns true if there is no more input to process.
func (i *Iter) Done() bool {
return i.p >= i.rb.nsrc
}
// Next returns f(i.input[i.Pos():n]), where n is a boundary of i.input.
// For any input a and b for which f(a) == f(b), subsequent calls
// to Next will return the same segments.
// Modifying runes are grouped together with the preceding starter, if such a starter exists.
// Although not guaranteed, n will typically be the smallest possible n.
func (i *Iter) Next() []byte {
return i.next(i)
}
func nextASCIIBytes(i *Iter) []byte {
p := i.p + 1
if p >= i.rb.nsrc {
i.setDone()
return i.rb.src.bytes[i.p:p]
}
if i.rb.src.bytes[p] < utf8.RuneSelf {
p0 := i.p
i.p = p
return i.rb.src.bytes[p0:p]
}
i.info = i.rb.f.info(i.rb.src, i.p)
i.next = i.rb.f.nextMain
return i.next(i)
}
func nextASCIIString(i *Iter) []byte {
p := i.p + 1
if p >= i.rb.nsrc {
i.buf[0] = i.rb.src.str[i.p]
i.setDone()
return i.buf[:1]
}
if i.rb.src.str[p] < utf8.RuneSelf {
i.buf[0] = i.rb.src.str[i.p]
i.p = p
return i.buf[:1]
}
i.info = i.rb.f.info(i.rb.src, i.p)
i.next = i.rb.f.nextMain
return i.next(i)
}
func nextHangul(i *Iter) []byte {
p := i.p
next := p + hangulUTF8Size
if next >= i.rb.nsrc {
i.setDone()
} else if i.rb.src.hangul(next) == 0 {
i.info = i.rb.f.info(i.rb.src, i.p)
i.next = i.rb.f.nextMain
return i.next(i)
}
i.p = next
return i.buf[:decomposeHangul(i.buf[:], i.rb.src.hangul(p))]
}
func nextDone(i *Iter) []byte {
return nil
}
// nextMulti is used for iterating over multi-segment decompositions
// for decomposing normal forms.
func nextMulti(i *Iter) []byte {
j := 0
d := i.multiSeg
// skip first rune
for j = 1; j < len(d) && !utf8.RuneStart(d[j]); j++ {
}
for j < len(d) {
info := i.rb.f.info(input{bytes: d}, j)
if info.BoundaryBefore() {
i.multiSeg = d[j:]
return d[:j]
}
j += int(info.size)
}
// treat last segment as normal decomposition
i.next = i.rb.f.nextMain
return i.next(i)
}
// nextMultiNorm is used for iterating over multi-segment decompositions
// for composing normal forms.
func nextMultiNorm(i *Iter) []byte {
j := 0
d := i.multiSeg
for j < len(d) {
info := i.rb.f.info(input{bytes: d}, j)
if info.BoundaryBefore() {
i.rb.compose()
seg := i.buf[:i.rb.flushCopy(i.buf[:])]
i.rb.ss.first(info)
i.rb.insertUnsafe(input{bytes: d}, j, info)
i.multiSeg = d[j+int(info.size):]
return seg
}
i.rb.ss.next(info)
i.rb.insertUnsafe(input{bytes: d}, j, info)
j += int(info.size)
}
i.multiSeg = nil
i.next = nextComposed
return doNormComposed(i)
}
// nextDecomposed is the implementation of Next for forms NFD and NFKD.
func nextDecomposed(i *Iter) (next []byte) {
outp := 0
inCopyStart, outCopyStart := i.p, 0
ss := mkStreamSafe(i.info)
for {
if sz := int(i.info.size); sz <= 1 {
p := i.p
i.p++ // ASCII or illegal byte. Either way, advance by 1.
if i.p >= i.rb.nsrc {
i.setDone()
return i.returnSlice(p, i.p)
} else if i.rb.src._byte(i.p) < utf8.RuneSelf {
i.next = i.asciiF
return i.returnSlice(p, i.p)
}
outp++
} else if d := i.info.Decomposition(); d != nil {
// Note: If leading CCC != 0, then len(d) == 2 and last is also non-zero.
// Case 1: there is a leftover to copy. In this case the decomposition
// must begin with a modifier and should always be appended.
// Case 2: no leftover. Simply return d if followed by a ccc == 0 value.
p := outp + len(d)
if outp > 0 {
i.rb.src.copySlice(i.buf[outCopyStart:], inCopyStart, i.p)
if p > len(i.buf) {
return i.buf[:outp]
}
} else if i.info.multiSegment() {
// outp must be 0 as multi-segment decompositions always
// start a new segment.
if i.multiSeg == nil {
i.multiSeg = d
i.next = nextMulti
return nextMulti(i)
}
// We are in the last segment. Treat as normal decomposition.
d = i.multiSeg
i.multiSeg = nil
p = len(d)
}
prevCC := i.info.tccc
if i.p += sz; i.p >= i.rb.nsrc {
i.setDone()
i.info = Properties{} // Force BoundaryBefore to succeed.
} else {
i.info = i.rb.f.info(i.rb.src, i.p)
}
switch ss.next(i.info) {
case ssOverflow:
i.next = nextCGJDecompose
fallthrough
case ssStarter:
if outp > 0 {
copy(i.buf[outp:], d)
return i.buf[:p]
}
return d
}
copy(i.buf[outp:], d)
outp = p
inCopyStart, outCopyStart = i.p, outp
if i.info.ccc < prevCC {
goto doNorm
}
continue
} else if r := i.rb.src.hangul(i.p); r != 0 {
outp = decomposeHangul(i.buf[:], r)
i.p += hangulUTF8Size
inCopyStart, outCopyStart = i.p, outp
if i.p >= i.rb.nsrc {
i.setDone()
break
} else if i.rb.src.hangul(i.p) != 0 {
i.next = nextHangul
return i.buf[:outp]
}
} else {
p := outp + sz
if p > len(i.buf) {
break
}
outp = p
i.p += sz
}
if i.p >= i.rb.nsrc {
i.setDone()
break
}
prevCC := i.info.tccc
i.info = i.rb.f.info(i.rb.src, i.p)
if v := ss.next(i.info); v == ssStarter {
break
} else if v == ssOverflow {
i.next = nextCGJDecompose
break
}
if i.info.ccc < prevCC {
goto doNorm
}
}
if outCopyStart == 0 {
return i.returnSlice(inCopyStart, i.p)
} else if inCopyStart < i.p {
i.rb.src.copySlice(i.buf[outCopyStart:], inCopyStart, i.p)
}
return i.buf[:outp]
doNorm:
// Insert what we have decomposed so far in the reorderBuffer.
// As we will only reorder, there will always be enough room.
i.rb.src.copySlice(i.buf[outCopyStart:], inCopyStart, i.p)
i.rb.insertDecomposed(i.buf[0:outp])
return doNormDecomposed(i)
}
func doNormDecomposed(i *Iter) []byte {
for {
if s := i.rb.ss.next(i.info); s == ssOverflow {
i.next = nextCGJDecompose
break
}
i.rb.insertUnsafe(i.rb.src, i.p, i.info)
if i.p += int(i.info.size); i.p >= i.rb.nsrc {
i.setDone()
break
}
i.info = i.rb.f.info(i.rb.src, i.p)
if i.info.ccc == 0 {
break
}
}
// new segment or too many combining characters: exit normalization
return i.buf[:i.rb.flushCopy(i.buf[:])]
}
func nextCGJDecompose(i *Iter) []byte {
i.rb.ss = 0
i.rb.insertCGJ()
i.next = nextDecomposed
buf := doNormDecomposed(i)
return buf
}
// nextComposed is the implementation of Next for forms NFC and NFKC.
func nextComposed(i *Iter) []byte {
outp, startp := 0, i.p
var prevCC uint8
ss := mkStreamSafe(i.info)
for {
if !i.info.isYesC() {
goto doNorm
}
prevCC = i.info.tccc
sz := int(i.info.size)
if sz == 0 {
sz = 1 // illegal rune: copy byte-by-byte
}
p := outp + sz
if p > len(i.buf) {
break
}
outp = p
i.p += sz
if i.p >= i.rb.nsrc {
i.setDone()
break
} else if i.rb.src._byte(i.p) < utf8.RuneSelf {
i.next = i.asciiF
break
}
i.info = i.rb.f.info(i.rb.src, i.p)
if v := ss.next(i.info); v == ssStarter {
break
} else if v == ssOverflow {
i.next = nextCGJCompose
break
}
if i.info.ccc < prevCC {
goto doNorm
}
}
return i.returnSlice(startp, i.p)
doNorm:
i.p = startp
i.info = i.rb.f.info(i.rb.src, i.p)
if i.info.multiSegment() {
d := i.info.Decomposition()
info := i.rb.f.info(input{bytes: d}, 0)
i.rb.insertUnsafe(input{bytes: d}, 0, info)
i.multiSeg = d[int(info.size):]
i.next = nextMultiNorm
return nextMultiNorm(i)
}
i.rb.ss.first(i.info)
i.rb.insertUnsafe(i.rb.src, i.p, i.info)
return doNormComposed(i)
}
func doNormComposed(i *Iter) []byte {
// First rune should already be inserted.
for {
if i.p += int(i.info.size); i.p >= i.rb.nsrc {
i.setDone()
break
}
i.info = i.rb.f.info(i.rb.src, i.p)
if s := i.rb.ss.next(i.info); s == ssStarter {
break
} else if s == ssOverflow {
i.next = nextCGJCompose
break
}
i.rb.insertUnsafe(i.rb.src, i.p, i.info)
}
i.rb.compose()
seg := i.buf[:i.rb.flushCopy(i.buf[:])]
return seg
}
func nextCGJCompose(i *Iter) []byte {
i.rb.ss = 0 // instead of first
i.rb.insertCGJ()
i.next = nextComposed
// Note that we treat any rune with nLeadingNonStarters > 0 as a non-starter,
// even if they are not. This is particularly dubious for U+FF9E and UFF9A.
// If we ever change that, insert a check here.
i.rb.ss.first(i.info)
i.rb.insertUnsafe(i.rb.src, i.p, i.info)
return doNormComposed(i)
}

View File

@@ -0,0 +1,98 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"strings"
"testing"
)
func doIterNorm(f Form, s string) []byte {
acc := []byte{}
i := Iter{}
i.InitString(f, s)
for !i.Done() {
acc = append(acc, i.Next()...)
}
return acc
}
func TestIterNext(t *testing.T) {
runNormTests(t, "IterNext", func(f Form, out []byte, s string) []byte {
return doIterNorm(f, string(append(out, s...)))
})
}
type SegmentTest struct {
in string
out []string
}
var segmentTests = []SegmentTest{
{"\u1E0A\u0323a", []string{"\x44\u0323\u0307", "a", ""}},
{rep('a', segSize), append(strings.Split(rep('a', segSize), ""), "")},
{rep('a', segSize+2), append(strings.Split(rep('a', segSize+2), ""), "")},
{rep('a', segSize) + "\u0300aa",
append(strings.Split(rep('a', segSize-1), ""), "a\u0300", "a", "a", "")},
// U+0f73 is NOT treated as a starter as it is a modifier
{"a" + grave(29) + "\u0f73", []string{"a" + grave(29), cgj + "\u0f73"}},
{"a\u0f73", []string{"a\u0f73"}},
// U+ff9e is treated as a non-starter.
// TODO: should we? Note that this will only affect iteration, as whether
// or not we do so does not affect the normalization output and will either
// way result in consistent iteration output.
{"a" + grave(30) + "\uff9e", []string{"a" + grave(30), cgj + "\uff9e"}},
{"a\uff9e", []string{"a\uff9e"}},
}
var segmentTestsK = []SegmentTest{
{"\u3332", []string{"\u30D5", "\u30A1", "\u30E9", "\u30C3", "\u30C8\u3099", ""}},
// last segment of multi-segment decomposition needs normalization
{"\u3332\u093C", []string{"\u30D5", "\u30A1", "\u30E9", "\u30C3", "\u30C8\u093C\u3099", ""}},
{"\u320E", []string{"\x28", "\uAC00", "\x29"}},
// last segment should be copied to start of buffer.
{"\ufdfa", []string{"\u0635", "\u0644", "\u0649", " ", "\u0627", "\u0644", "\u0644", "\u0647", " ", "\u0639", "\u0644", "\u064a", "\u0647", " ", "\u0648", "\u0633", "\u0644", "\u0645", ""}},
{"\ufdfa" + grave(30), []string{"\u0635", "\u0644", "\u0649", " ", "\u0627", "\u0644", "\u0644", "\u0647", " ", "\u0639", "\u0644", "\u064a", "\u0647", " ", "\u0648", "\u0633", "\u0644", "\u0645" + grave(30), ""}},
{"\uFDFA" + grave(64), []string{"\u0635", "\u0644", "\u0649", " ", "\u0627", "\u0644", "\u0644", "\u0647", " ", "\u0639", "\u0644", "\u064a", "\u0647", " ", "\u0648", "\u0633", "\u0644", "\u0645" + grave(30), cgj + grave(30), cgj + grave(4), ""}},
// Hangul and Jamo are grouped togeter.
{"\uAC00", []string{"\u1100\u1161", ""}},
{"\uAC01", []string{"\u1100\u1161\u11A8", ""}},
{"\u1100\u1161", []string{"\u1100\u1161", ""}},
}
// Note that, by design, segmentation is equal for composing and decomposing forms.
func TestIterSegmentation(t *testing.T) {
segmentTest(t, "SegmentTestD", NFD, segmentTests)
segmentTest(t, "SegmentTestC", NFC, segmentTests)
segmentTest(t, "SegmentTestKD", NFKD, segmentTestsK)
segmentTest(t, "SegmentTestKC", NFKC, segmentTestsK)
}
func segmentTest(t *testing.T, name string, f Form, tests []SegmentTest) {
iter := Iter{}
for i, tt := range tests {
iter.InitString(f, tt.in)
for j, seg := range tt.out {
if seg == "" {
if !iter.Done() {
res := string(iter.Next())
t.Errorf(`%s:%d:%d: expected Done()==true, found segment %+q`, name, i, j, res)
}
continue
}
if iter.Done() {
t.Errorf("%s:%d:%d: Done()==true, want false", name, i, j)
}
seg = f.String(seg)
if res := string(iter.Next()); res != seg {
t.Errorf(`%s:%d:%d" segment was %+q (%d); want %+q (%d)`, name, i, j, pc(res), len(res), pc(seg), len(seg))
}
}
}
}

View File

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,45 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
// Generate test data for trie code.
package main
import (
"fmt"
)
func main() {
printTestTables()
}
// We take the smallest, largest and an arbitrary value for each
// of the UTF-8 sequence lengths.
var testRunes = []rune{
0x01, 0x0C, 0x7F, // 1-byte sequences
0x80, 0x100, 0x7FF, // 2-byte sequences
0x800, 0x999, 0xFFFF, // 3-byte sequences
0x10000, 0x10101, 0x10FFFF, // 4-byte sequences
0x200, 0x201, 0x202, 0x210, 0x215, // five entries in one sparse block
}
const fileHeader = `// Generated by running
// maketesttables
// DO NOT EDIT
package norm
`
func printTestTables() {
fmt.Print(fileHeader)
fmt.Printf("var testRunes = %#v\n\n", testRunes)
t := newNode()
for i, r := range testRunes {
t.insert(r, uint16(i))
}
t.printTables("testdata")
}

View File

@@ -0,0 +1,14 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm_test
import (
"testing"
)
func TestPlaceHolder(t *testing.T) {
// Does nothing, just allows the Makefile to be canonical
// while waiting for the package itself to be written.
}

View File

@@ -0,0 +1,524 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package norm contains types and functions for normalizing Unicode strings.
package norm
import "unicode/utf8"
// A Form denotes a canonical representation of Unicode code points.
// The Unicode-defined normalization and equivalence forms are:
//
// NFC Unicode Normalization Form C
// NFD Unicode Normalization Form D
// NFKC Unicode Normalization Form KC
// NFKD Unicode Normalization Form KD
//
// For a Form f, this documentation uses the notation f(x) to mean
// the bytes or string x converted to the given form.
// A position n in x is called a boundary if conversion to the form can
// proceed independently on both sides:
// f(x) == append(f(x[0:n]), f(x[n:])...)
//
// References: http://unicode.org/reports/tr15/ and
// http://unicode.org/notes/tn5/.
type Form int
const (
NFC Form = iota
NFD
NFKC
NFKD
)
// Bytes returns f(b). May return b if f(b) = b.
func (f Form) Bytes(b []byte) []byte {
src := inputBytes(b)
ft := formTable[f]
n, ok := ft.quickSpan(src, 0, len(b), true)
if ok {
return b
}
out := make([]byte, n, len(b))
copy(out, b[0:n])
rb := reorderBuffer{f: *ft, src: src, nsrc: len(b), out: out, flushF: appendFlush}
return doAppendInner(&rb, n)
}
// String returns f(s).
func (f Form) String(s string) string {
src := inputString(s)
ft := formTable[f]
n, ok := ft.quickSpan(src, 0, len(s), true)
if ok {
return s
}
out := make([]byte, n, len(s))
copy(out, s[0:n])
rb := reorderBuffer{f: *ft, src: src, nsrc: len(s), out: out, flushF: appendFlush}
return string(doAppendInner(&rb, n))
}
// IsNormal returns true if b == f(b).
func (f Form) IsNormal(b []byte) bool {
src := inputBytes(b)
ft := formTable[f]
bp, ok := ft.quickSpan(src, 0, len(b), true)
if ok {
return true
}
rb := reorderBuffer{f: *ft, src: src, nsrc: len(b)}
rb.setFlusher(nil, cmpNormalBytes)
for bp < len(b) {
rb.out = b[bp:]
if bp = decomposeSegment(&rb, bp, true); bp < 0 {
return false
}
bp, _ = rb.f.quickSpan(rb.src, bp, len(b), true)
}
return true
}
func cmpNormalBytes(rb *reorderBuffer) bool {
b := rb.out
for i := 0; i < rb.nrune; i++ {
info := rb.rune[i]
if int(info.size) > len(b) {
return false
}
p := info.pos
pe := p + info.size
for ; p < pe; p++ {
if b[0] != rb.byte[p] {
return false
}
b = b[1:]
}
}
return true
}
// IsNormalString returns true if s == f(s).
func (f Form) IsNormalString(s string) bool {
src := inputString(s)
ft := formTable[f]
bp, ok := ft.quickSpan(src, 0, len(s), true)
if ok {
return true
}
rb := reorderBuffer{f: *ft, src: src, nsrc: len(s)}
rb.setFlusher(nil, func(rb *reorderBuffer) bool {
for i := 0; i < rb.nrune; i++ {
info := rb.rune[i]
if bp+int(info.size) > len(s) {
return false
}
p := info.pos
pe := p + info.size
for ; p < pe; p++ {
if s[bp] != rb.byte[p] {
return false
}
bp++
}
}
return true
})
for bp < len(s) {
if bp = decomposeSegment(&rb, bp, true); bp < 0 {
return false
}
bp, _ = rb.f.quickSpan(rb.src, bp, len(s), true)
}
return true
}
// patchTail fixes a case where a rune may be incorrectly normalized
// if it is followed by illegal continuation bytes. It returns the
// patched buffer and whether the decomposition is still in progress.
func patchTail(rb *reorderBuffer) bool {
info, p := lastRuneStart(&rb.f, rb.out)
if p == -1 || info.size == 0 {
return true
}
end := p + int(info.size)
extra := len(rb.out) - end
if extra > 0 {
// Potentially allocating memory. However, this only
// happens with ill-formed UTF-8.
x := make([]byte, 0)
x = append(x, rb.out[len(rb.out)-extra:]...)
rb.out = rb.out[:end]
decomposeToLastBoundary(rb)
rb.doFlush()
rb.out = append(rb.out, x...)
return false
}
buf := rb.out[p:]
rb.out = rb.out[:p]
decomposeToLastBoundary(rb)
if s := rb.ss.next(info); s == ssStarter {
rb.doFlush()
rb.ss.first(info)
} else if s == ssOverflow {
rb.doFlush()
rb.insertCGJ()
rb.ss = 0
}
rb.insertUnsafe(inputBytes(buf), 0, info)
return true
}
func appendQuick(rb *reorderBuffer, i int) int {
if rb.nsrc == i {
return i
}
end, _ := rb.f.quickSpan(rb.src, i, rb.nsrc, true)
rb.out = rb.src.appendSlice(rb.out, i, end)
return end
}
// Append returns f(append(out, b...)).
// The buffer out must be nil, empty, or equal to f(out).
func (f Form) Append(out []byte, src ...byte) []byte {
return f.doAppend(out, inputBytes(src), len(src))
}
func (f Form) doAppend(out []byte, src input, n int) []byte {
if n == 0 {
return out
}
ft := formTable[f]
// Attempt to do a quickSpan first so we can avoid initializing the reorderBuffer.
if len(out) == 0 {
p, _ := ft.quickSpan(src, 0, n, true)
out = src.appendSlice(out, 0, p)
if p == n {
return out
}
rb := reorderBuffer{f: *ft, src: src, nsrc: n, out: out, flushF: appendFlush}
return doAppendInner(&rb, p)
}
rb := reorderBuffer{f: *ft, src: src, nsrc: n}
return doAppend(&rb, out, 0)
}
func doAppend(rb *reorderBuffer, out []byte, p int) []byte {
rb.setFlusher(out, appendFlush)
src, n := rb.src, rb.nsrc
doMerge := len(out) > 0
if q := src.skipContinuationBytes(p); q > p {
// Move leading non-starters to destination.
rb.out = src.appendSlice(rb.out, p, q)
p = q
doMerge = patchTail(rb)
}
fd := &rb.f
if doMerge {
var info Properties
if p < n {
info = fd.info(src, p)
if !info.BoundaryBefore() || info.nLeadingNonStarters() > 0 {
if p == 0 {
decomposeToLastBoundary(rb)
}
p = decomposeSegment(rb, p, true)
}
}
if info.size == 0 {
rb.doFlush()
// Append incomplete UTF-8 encoding.
return src.appendSlice(rb.out, p, n)
}
if rb.nrune > 0 {
return doAppendInner(rb, p)
}
}
p = appendQuick(rb, p)
return doAppendInner(rb, p)
}
func doAppendInner(rb *reorderBuffer, p int) []byte {
for n := rb.nsrc; p < n; {
p = decomposeSegment(rb, p, true)
p = appendQuick(rb, p)
}
return rb.out
}
// AppendString returns f(append(out, []byte(s))).
// The buffer out must be nil, empty, or equal to f(out).
func (f Form) AppendString(out []byte, src string) []byte {
return f.doAppend(out, inputString(src), len(src))
}
// QuickSpan returns a boundary n such that b[0:n] == f(b[0:n]).
// It is not guaranteed to return the largest such n.
func (f Form) QuickSpan(b []byte) int {
n, _ := formTable[f].quickSpan(inputBytes(b), 0, len(b), true)
return n
}
// quickSpan returns a boundary n such that src[0:n] == f(src[0:n]) and
// whether any non-normalized parts were found. If atEOF is false, n will
// not point past the last segment if this segment might be become
// non-normalized by appending other runes.
func (f *formInfo) quickSpan(src input, i, end int, atEOF bool) (n int, ok bool) {
var lastCC uint8
ss := streamSafe(0)
lastSegStart := i
for n = end; i < n; {
if j := src.skipASCII(i, n); i != j {
i = j
lastSegStart = i - 1
lastCC = 0
ss = 0
continue
}
info := f.info(src, i)
if info.size == 0 {
if atEOF {
// include incomplete runes
return n, true
}
return lastSegStart, true
}
// This block needs to be before the next, because it is possible to
// have an overflow for runes that are starters (e.g. with U+FF9E).
switch ss.next(info) {
case ssStarter:
ss.first(info)
lastSegStart = i
case ssOverflow:
return lastSegStart, false
case ssSuccess:
if lastCC > info.ccc {
return lastSegStart, false
}
}
if f.composing {
if !info.isYesC() {
break
}
} else {
if !info.isYesD() {
break
}
}
lastCC = info.ccc
i += int(info.size)
}
if i == n {
if !atEOF {
n = lastSegStart
}
return n, true
}
return lastSegStart, false
}
// QuickSpanString returns a boundary n such that b[0:n] == f(s[0:n]).
// It is not guaranteed to return the largest such n.
func (f Form) QuickSpanString(s string) int {
n, _ := formTable[f].quickSpan(inputString(s), 0, len(s), true)
return n
}
// FirstBoundary returns the position i of the first boundary in b
// or -1 if b contains no boundary.
func (f Form) FirstBoundary(b []byte) int {
return f.firstBoundary(inputBytes(b), len(b))
}
func (f Form) firstBoundary(src input, nsrc int) int {
i := src.skipContinuationBytes(0)
if i >= nsrc {
return -1
}
fd := formTable[f]
ss := streamSafe(0)
// We should call ss.first here, but we can't as the first rune is
// skipped already. This means FirstBoundary can't really determine
// CGJ insertion points correctly. Luckily it doesn't have to.
// TODO: consider adding NextBoundary
for {
info := fd.info(src, i)
if info.size == 0 {
return -1
}
if s := ss.next(info); s != ssSuccess {
return i
}
i += int(info.size)
if i >= nsrc {
if !info.BoundaryAfter() && !ss.isMax() {
return -1
}
return nsrc
}
}
}
// FirstBoundaryInString returns the position i of the first boundary in s
// or -1 if s contains no boundary.
func (f Form) FirstBoundaryInString(s string) int {
return f.firstBoundary(inputString(s), len(s))
}
// LastBoundary returns the position i of the last boundary in b
// or -1 if b contains no boundary.
func (f Form) LastBoundary(b []byte) int {
return lastBoundary(formTable[f], b)
}
func lastBoundary(fd *formInfo, b []byte) int {
i := len(b)
info, p := lastRuneStart(fd, b)
if p == -1 {
return -1
}
if info.size == 0 { // ends with incomplete rune
if p == 0 { // starts with incomplete rune
return -1
}
i = p
info, p = lastRuneStart(fd, b[:i])
if p == -1 { // incomplete UTF-8 encoding or non-starter bytes without a starter
return i
}
}
if p+int(info.size) != i { // trailing non-starter bytes: illegal UTF-8
return i
}
if info.BoundaryAfter() {
return i
}
ss := streamSafe(0)
v := ss.backwards(info)
for i = p; i >= 0 && v != ssStarter; i = p {
info, p = lastRuneStart(fd, b[:i])
if v = ss.backwards(info); v == ssOverflow {
break
}
if p+int(info.size) != i {
if p == -1 { // no boundary found
return -1
}
return i // boundary after an illegal UTF-8 encoding
}
}
return i
}
// decomposeSegment scans the first segment in src into rb. It inserts 0x034f
// (Grapheme Joiner) when it encounters a sequence of more than 30 non-starters
// and returns the number of bytes consumed from src or iShortDst or iShortSrc.
func decomposeSegment(rb *reorderBuffer, sp int, atEOF bool) int {
// Force one character to be consumed.
info := rb.f.info(rb.src, sp)
if info.size == 0 {
return 0
}
if rb.nrune > 0 {
if s := rb.ss.next(info); s == ssStarter {
goto end
} else if s == ssOverflow {
rb.insertCGJ()
goto end
}
} else {
rb.ss.first(info)
}
if err := rb.insertFlush(rb.src, sp, info); err != iSuccess {
return int(err)
}
for {
sp += int(info.size)
if sp >= rb.nsrc {
if !atEOF && !info.BoundaryAfter() {
return int(iShortSrc)
}
break
}
info = rb.f.info(rb.src, sp)
if info.size == 0 {
if !atEOF {
return int(iShortSrc)
}
break
}
if s := rb.ss.next(info); s == ssStarter {
break
} else if s == ssOverflow {
rb.insertCGJ()
break
}
if err := rb.insertFlush(rb.src, sp, info); err != iSuccess {
return int(err)
}
}
end:
if !rb.doFlush() {
return int(iShortDst)
}
return sp
}
// lastRuneStart returns the runeInfo and position of the last
// rune in buf or the zero runeInfo and -1 if no rune was found.
func lastRuneStart(fd *formInfo, buf []byte) (Properties, int) {
p := len(buf) - 1
for ; p >= 0 && !utf8.RuneStart(buf[p]); p-- {
}
if p < 0 {
return Properties{}, -1
}
return fd.info(inputBytes(buf), p), p
}
// decomposeToLastBoundary finds an open segment at the end of the buffer
// and scans it into rb. Returns the buffer minus the last segment.
func decomposeToLastBoundary(rb *reorderBuffer) {
fd := &rb.f
info, i := lastRuneStart(fd, rb.out)
if int(info.size) != len(rb.out)-i {
// illegal trailing continuation bytes
return
}
if info.BoundaryAfter() {
return
}
var add [maxNonStarters + 1]Properties // stores runeInfo in reverse order
padd := 0
ss := streamSafe(0)
p := len(rb.out)
for {
add[padd] = info
v := ss.backwards(info)
if v == ssOverflow {
// Note that if we have an overflow, it the string we are appending to
// is not correctly normalized. In this case the behavior is undefined.
break
}
padd++
p -= int(info.size)
if v == ssStarter || p < 0 {
break
}
info, i = lastRuneStart(fd, rb.out[:p])
if int(info.size) != p-i {
break
}
}
rb.ss = ss
// Copy bytes for insertion as we may need to overwrite rb.out.
var buf [maxBufferSize * utf8.UTFMax]byte
cp := buf[:copy(buf[:], rb.out[p:])]
rb.out = rb.out[:p]
for padd--; padd >= 0; padd-- {
info = add[padd]
rb.insertUnsafe(inputBytes(cp), 0, info)
cp = cp[info.size:]
}
}

View File

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,318 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
package main
import (
"bufio"
"bytes"
"flag"
"fmt"
"log"
"net/http"
"os"
"path"
"regexp"
"runtime"
"strconv"
"strings"
"time"
"unicode"
"unicode/utf8"
"code.google.com/p/go.text/unicode/norm"
)
func main() {
flag.Parse()
loadTestData()
CharacterByCharacterTests()
StandardTests()
PerformanceTest()
if errorCount == 0 {
fmt.Println("PASS")
}
}
const file = "NormalizationTest.txt"
var url = flag.String("url",
"http://www.unicode.org/Public/"+unicode.Version+"/ucd/"+file,
"URL of Unicode database directory")
var localFiles = flag.Bool("local",
false,
"data files have been copied to the current directory; for debugging only")
var logger = log.New(os.Stderr, "", log.Lshortfile)
// This regression test runs the test set in NormalizationTest.txt
// (taken from http://www.unicode.org/Public/<unicode.Version>/ucd/).
//
// NormalizationTest.txt has form:
// @Part0 # Specific cases
// #
// 1E0A;1E0A;0044 0307;1E0A;0044 0307; # (Ḋ; Ḋ; D◌̇; Ḋ; D◌̇; ) LATIN CAPITAL LETTER D WITH DOT ABOVE
// 1E0C;1E0C;0044 0323;1E0C;0044 0323; # (Ḍ; Ḍ; D◌̣; Ḍ; D◌̣; ) LATIN CAPITAL LETTER D WITH DOT BELOW
//
// Each test has 5 columns (c1, c2, c3, c4, c5), where
// (c1, c2, c3, c4, c5) == (c1, NFC(c1), NFD(c1), NFKC(c1), NFKD(c1))
//
// CONFORMANCE:
// 1. The following invariants must be true for all conformant implementations
//
// NFC
// c2 == NFC(c1) == NFC(c2) == NFC(c3)
// c4 == NFC(c4) == NFC(c5)
//
// NFD
// c3 == NFD(c1) == NFD(c2) == NFD(c3)
// c5 == NFD(c4) == NFD(c5)
//
// NFKC
// c4 == NFKC(c1) == NFKC(c2) == NFKC(c3) == NFKC(c4) == NFKC(c5)
//
// NFKD
// c5 == NFKD(c1) == NFKD(c2) == NFKD(c3) == NFKD(c4) == NFKD(c5)
//
// 2. For every code point X assigned in this version of Unicode that is not
// specifically listed in Part 1, the following invariants must be true
// for all conformant implementations:
//
// X == NFC(X) == NFD(X) == NFKC(X) == NFKD(X)
//
// Column types.
const (
cRaw = iota
cNFC
cNFD
cNFKC
cNFKD
cMaxColumns
)
// Holds data from NormalizationTest.txt
var part []Part
type Part struct {
name string
number int
tests []Test
}
type Test struct {
name string
partnr int
number int
r rune // used for character by character test
cols [cMaxColumns]string // Each has 5 entries, see below.
}
func (t Test) Name() string {
if t.number < 0 {
return part[t.partnr].name
}
return fmt.Sprintf("%s:%d", part[t.partnr].name, t.number)
}
var partRe = regexp.MustCompile(`@Part(\d) # (.*)$`)
var testRe = regexp.MustCompile(`^` + strings.Repeat(`([\dA-F ]+);`, 5) + ` # (.*)$`)
var counter int
// Load the data form NormalizationTest.txt
func loadTestData() {
if *localFiles {
pwd, _ := os.Getwd()
*url = "file://" + path.Join(pwd, file)
}
t := &http.Transport{}
t.RegisterProtocol("file", http.NewFileTransport(http.Dir("/")))
c := &http.Client{Transport: t}
resp, err := c.Get(*url)
if err != nil {
logger.Fatal(err)
}
if resp.StatusCode != 200 {
logger.Fatal("bad GET status for "+file, resp.Status)
}
f := resp.Body
defer f.Close()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
line := scanner.Text()
if len(line) == 0 || line[0] == '#' {
continue
}
m := partRe.FindStringSubmatch(line)
if m != nil {
if len(m) < 3 {
logger.Fatal("Failed to parse Part: ", line)
}
i, err := strconv.Atoi(m[1])
if err != nil {
logger.Fatal(err)
}
name := m[2]
part = append(part, Part{name: name[:len(name)-1], number: i})
continue
}
m = testRe.FindStringSubmatch(line)
if m == nil || len(m) < 7 {
logger.Fatalf(`Failed to parse: "%s" result: %#v`, line, m)
}
test := Test{name: m[6], partnr: len(part) - 1, number: counter}
counter++
for j := 1; j < len(m)-1; j++ {
for _, split := range strings.Split(m[j], " ") {
r, err := strconv.ParseUint(split, 16, 64)
if err != nil {
logger.Fatal(err)
}
if test.r == 0 {
// save for CharacterByCharacterTests
test.r = rune(r)
}
var buf [utf8.UTFMax]byte
sz := utf8.EncodeRune(buf[:], rune(r))
test.cols[j-1] += string(buf[:sz])
}
}
part := &part[len(part)-1]
part.tests = append(part.tests, test)
}
if scanner.Err() != nil {
logger.Fatal(scanner.Err())
}
}
var fstr = []string{"NFC", "NFD", "NFKC", "NFKD"}
var errorCount int
func cmpResult(t *Test, name string, f norm.Form, gold, test, result string) {
if gold != result {
errorCount++
if errorCount > 20 {
return
}
logger.Printf("%s:%s: %s(%+q)=%+q; want %+q: %s",
t.Name(), name, fstr[f], test, result, gold, t.name)
}
}
func cmpIsNormal(t *Test, name string, f norm.Form, test string, result, want bool) {
if result != want {
errorCount++
if errorCount > 20 {
return
}
logger.Printf("%s:%s: %s(%+q)=%v; want %v", t.Name(), name, fstr[f], test, result, want)
}
}
func doTest(t *Test, f norm.Form, gold, test string) {
testb := []byte(test)
result := f.Bytes(testb)
cmpResult(t, "Bytes", f, gold, test, string(result))
sresult := f.String(test)
cmpResult(t, "String", f, gold, test, sresult)
acc := []byte{}
i := norm.Iter{}
i.InitString(f, test)
for !i.Done() {
acc = append(acc, i.Next()...)
}
cmpResult(t, "Iter.Next", f, gold, test, string(acc))
buf := make([]byte, 128)
acc = nil
for p := 0; p < len(testb); {
nDst, nSrc, _ := f.Transform(buf, testb[p:], true)
acc = append(acc, buf[:nDst]...)
p += nSrc
}
cmpResult(t, "Transform", f, gold, test, string(acc))
for i := range test {
out := f.Append(f.Bytes([]byte(test[:i])), []byte(test[i:])...)
cmpResult(t, fmt.Sprintf(":Append:%d", i), f, gold, test, string(out))
}
cmpIsNormal(t, "IsNormal", f, test, f.IsNormal([]byte(test)), test == gold)
cmpIsNormal(t, "IsNormalString", f, test, f.IsNormalString(test), test == gold)
}
func doConformanceTests(t *Test, partn int) {
for i := 0; i <= 2; i++ {
doTest(t, norm.NFC, t.cols[1], t.cols[i])
doTest(t, norm.NFD, t.cols[2], t.cols[i])
doTest(t, norm.NFKC, t.cols[3], t.cols[i])
doTest(t, norm.NFKD, t.cols[4], t.cols[i])
}
for i := 3; i <= 4; i++ {
doTest(t, norm.NFC, t.cols[3], t.cols[i])
doTest(t, norm.NFD, t.cols[4], t.cols[i])
doTest(t, norm.NFKC, t.cols[3], t.cols[i])
doTest(t, norm.NFKD, t.cols[4], t.cols[i])
}
}
func CharacterByCharacterTests() {
tests := part[1].tests
var last rune = 0
for i := 0; i <= len(tests); i++ { // last one is special case
var r rune
if i == len(tests) {
r = 0x2FA1E // Don't have to go to 0x10FFFF
} else {
r = tests[i].r
}
for last++; last < r; last++ {
// Check all characters that were not explicitly listed in the test.
t := &Test{partnr: 1, number: -1}
char := string(last)
doTest(t, norm.NFC, char, char)
doTest(t, norm.NFD, char, char)
doTest(t, norm.NFKC, char, char)
doTest(t, norm.NFKD, char, char)
}
if i < len(tests) {
doConformanceTests(&tests[i], 1)
}
}
}
func StandardTests() {
for _, j := range []int{0, 2, 3} {
for _, test := range part[j].tests {
doConformanceTests(&test, j)
}
}
}
// PerformanceTest verifies that normalization is O(n). If any of the
// code does not properly check for maxCombiningChars, normalization
// may exhibit O(n**2) behavior.
func PerformanceTest() {
runtime.GOMAXPROCS(2)
success := make(chan bool, 1)
go func() {
buf := bytes.Repeat([]byte("\u035D"), 1024*1024)
buf = append(buf, "\u035B"...)
norm.NFC.Append(nil, buf...)
success <- true
}()
timeout := time.After(1 * time.Second)
select {
case <-success:
// test completed before the timeout
case <-timeout:
errorCount++
logger.Printf(`unexpectedly long time to complete PerformanceTest`)
}
}

View File

@@ -0,0 +1,126 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import "io"
type normWriter struct {
rb reorderBuffer
w io.Writer
buf []byte
}
// Write implements the standard write interface. If the last characters are
// not at a normalization boundary, the bytes will be buffered for the next
// write. The remaining bytes will be written on close.
func (w *normWriter) Write(data []byte) (n int, err error) {
// Process data in pieces to keep w.buf size bounded.
const chunk = 4000
for len(data) > 0 {
// Normalize into w.buf.
m := len(data)
if m > chunk {
m = chunk
}
w.rb.src = inputBytes(data[:m])
w.rb.nsrc = m
w.buf = doAppend(&w.rb, w.buf, 0)
data = data[m:]
n += m
// Write out complete prefix, save remainder.
// Note that lastBoundary looks back at most 31 runes.
i := lastBoundary(&w.rb.f, w.buf)
if i == -1 {
i = 0
}
if i > 0 {
if _, err = w.w.Write(w.buf[:i]); err != nil {
break
}
bn := copy(w.buf, w.buf[i:])
w.buf = w.buf[:bn]
}
}
return n, err
}
// Close forces data that remains in the buffer to be written.
func (w *normWriter) Close() error {
if len(w.buf) > 0 {
_, err := w.w.Write(w.buf)
if err != nil {
return err
}
}
return nil
}
// Writer returns a new writer that implements Write(b)
// by writing f(b) to w. The returned writer may use an
// an internal buffer to maintain state across Write calls.
// Calling its Close method writes any buffered data to w.
func (f Form) Writer(w io.Writer) io.WriteCloser {
wr := &normWriter{rb: reorderBuffer{}, w: w}
wr.rb.init(f, nil)
return wr
}
type normReader struct {
rb reorderBuffer
r io.Reader
inbuf []byte
outbuf []byte
bufStart int
lastBoundary int
err error
}
// Read implements the standard read interface.
func (r *normReader) Read(p []byte) (int, error) {
for {
if r.lastBoundary-r.bufStart > 0 {
n := copy(p, r.outbuf[r.bufStart:r.lastBoundary])
r.bufStart += n
if r.lastBoundary-r.bufStart > 0 {
return n, nil
}
return n, r.err
}
if r.err != nil {
return 0, r.err
}
outn := copy(r.outbuf, r.outbuf[r.lastBoundary:])
r.outbuf = r.outbuf[0:outn]
r.bufStart = 0
n, err := r.r.Read(r.inbuf)
r.rb.src = inputBytes(r.inbuf[0:n])
r.rb.nsrc, r.err = n, err
if n > 0 {
r.outbuf = doAppend(&r.rb, r.outbuf, 0)
}
if err == io.EOF {
r.lastBoundary = len(r.outbuf)
} else {
r.lastBoundary = lastBoundary(&r.rb.f, r.outbuf)
if r.lastBoundary == -1 {
r.lastBoundary = 0
}
}
}
panic("should not reach here")
}
// Reader returns a new reader that implements Read
// by reading data from r and returning f(data).
func (f Form) Reader(r io.Reader) io.Reader {
const chunk = 4000
buf := make([]byte, chunk)
rr := &normReader{rb: reorderBuffer{}, r: r, inbuf: buf}
rr.rb.init(f, buf)
return rr
}

View File

@@ -0,0 +1,56 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"bytes"
"fmt"
"testing"
)
var bufSizes = []int{1, 2, 3, 4, 5, 6, 7, 8, 100, 101, 102, 103, 4000, 4001, 4002, 4003}
func readFunc(size int) appendFunc {
return func(f Form, out []byte, s string) []byte {
out = append(out, s...)
r := f.Reader(bytes.NewBuffer(out))
buf := make([]byte, size)
result := []byte{}
for n, err := 0, error(nil); err == nil; {
n, err = r.Read(buf)
result = append(result, buf[:n]...)
}
return result
}
}
func TestReader(t *testing.T) {
for _, s := range bufSizes {
name := fmt.Sprintf("TestReader%d", s)
runNormTests(t, name, readFunc(s))
}
}
func writeFunc(size int) appendFunc {
return func(f Form, out []byte, s string) []byte {
in := append(out, s...)
result := new(bytes.Buffer)
w := f.Writer(result)
buf := make([]byte, size)
for n := 0; len(in) > 0; in = in[n:] {
n = copy(buf, in)
_, _ = w.Write(buf[:n])
}
w.Close()
return result.Bytes()
}
}
func TestWriter(t *testing.T) {
for _, s := range bufSizes {
name := fmt.Sprintf("TestWriter%d", s)
runNormTests(t, name, writeFunc(s))
}
}

View File

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,85 @@
// Copyright 2013 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"unicode/utf8"
"code.google.com/p/go.text/transform"
)
// Transform implements the transform.Transformer interface. It may need to
// write segments of up to MaxSegmentSize at once. Users should either catch
// ErrShortDst and allow dst to grow or have dst be at least of size
// MaxTransformChunkSize to be guaranteed of progress.
func (f Form) Transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
n := 0
// Cap the maximum number of src bytes to check.
b := src
eof := atEOF
if ns := len(dst); ns < len(b) {
err = transform.ErrShortDst
eof = false
b = b[:ns]
}
i, ok := formTable[f].quickSpan(inputBytes(b), n, len(b), eof)
n += copy(dst[n:], b[n:i])
if !ok {
nDst, nSrc, err = f.transform(dst[n:], src[n:], atEOF)
return nDst + n, nSrc + n, err
}
if n < len(src) && !atEOF {
err = transform.ErrShortSrc
}
return n, n, err
}
func flushTransform(rb *reorderBuffer) bool {
// Write out (must fully fit in dst, or else it is a ErrShortDst).
if len(rb.out) < rb.nrune*utf8.UTFMax {
return false
}
rb.out = rb.out[rb.flushCopy(rb.out):]
return true
}
var errs = []error{nil, transform.ErrShortDst, transform.ErrShortSrc}
// transform implements the transform.Transformer interface. It is only called
// when quickSpan does not pass for a given string.
func (f Form) transform(dst, src []byte, atEOF bool) (nDst, nSrc int, err error) {
// TODO: get rid of reorderBuffer. See CL 23460044.
rb := reorderBuffer{}
rb.init(f, src)
for {
// Load segment into reorder buffer.
rb.setFlusher(dst[nDst:], flushTransform)
end := decomposeSegment(&rb, nSrc, atEOF)
if end < 0 {
return nDst, nSrc, errs[-end]
}
nDst = len(dst) - len(rb.out)
nSrc = end
// Next quickSpan.
end = rb.nsrc
eof := atEOF
if n := nSrc + len(dst) - nDst; n < end {
err = transform.ErrShortDst
end = n
eof = false
}
end, ok := rb.f.quickSpan(rb.src, nSrc, end, eof)
n := copy(dst[nDst:], rb.src.bytes[nSrc:end])
nSrc += n
nDst += n
if ok {
if n < rb.nsrc && !atEOF {
err = transform.ErrShortSrc
}
return nDst, nSrc, err
}
}
}

View File

@@ -0,0 +1,101 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"fmt"
"testing"
"code.google.com/p/go.text/transform"
)
func TestTransform(t *testing.T) {
tests := []struct {
f Form
in, out string
eof bool
dstSize int
err error
}{
{NFC, "ab", "ab", true, 2, nil},
{NFC, "qx", "qx", true, 2, nil},
{NFD, "qx", "qx", true, 2, nil},
{NFC, "", "", true, 1, nil},
{NFD, "", "", true, 1, nil},
{NFC, "", "", false, 1, nil},
{NFD, "", "", false, 1, nil},
// Normalized segment does not fit in destination.
{NFD, "ö", "", true, 1, transform.ErrShortDst},
{NFD, "ö", "", true, 2, transform.ErrShortDst},
// As an artifact of the algorithm, only full segments are written.
// This is not strictly required, and some bytes could be written.
// In practice, for Transform to not block, the destination buffer
// should be at least MaxSegmentSize to work anyway and these edge
// conditions will be relatively rare.
{NFC, "ab", "", true, 1, transform.ErrShortDst},
// This is even true for inert runes.
{NFC, "qx", "", true, 1, transform.ErrShortDst},
{NFC, "a\u0300abc", "\u00e0a", true, 4, transform.ErrShortDst},
// We cannot write a segment if succesive runes could still change the result.
{NFD, "ö", "", false, 3, transform.ErrShortSrc},
{NFC, "a\u0300", "", false, 4, transform.ErrShortSrc},
{NFD, "a\u0300", "", false, 4, transform.ErrShortSrc},
{NFC, "ö", "", false, 3, transform.ErrShortSrc},
{NFC, "a\u0300", "", true, 1, transform.ErrShortDst},
// Theoretically could fit, but won't due to simplified checks.
{NFC, "a\u0300", "", true, 2, transform.ErrShortDst},
{NFC, "a\u0300", "", true, 3, transform.ErrShortDst},
{NFC, "a\u0300", "\u00e0", true, 4, nil},
{NFD, "öa\u0300", "o\u0308", false, 8, transform.ErrShortSrc},
{NFD, "öa\u0300ö", "o\u0308a\u0300", true, 8, transform.ErrShortDst},
{NFD, "öa\u0300ö", "o\u0308a\u0300", false, 12, transform.ErrShortSrc},
// Illegal input is copied verbatim.
{NFD, "\xbd\xb2=\xbc ", "\xbd\xb2=\xbc ", true, 8, nil},
}
b := make([]byte, 100)
for i, tt := range tests {
nDst, _, err := tt.f.Transform(b[:tt.dstSize], []byte(tt.in), tt.eof)
out := string(b[:nDst])
if out != tt.out || err != tt.err {
t.Errorf("%d: was %+q (%v); want %+q (%v)", i, out, err, tt.out, tt.err)
}
if want := tt.f.String(tt.in)[:nDst]; want != out {
t.Errorf("%d: incorect normalization: was %+q; want %+q", i, out, want)
}
}
}
var transBufSizes = []int{
MaxTransformChunkSize,
3 * MaxTransformChunkSize / 2,
2 * MaxTransformChunkSize,
3 * MaxTransformChunkSize,
100 * MaxTransformChunkSize,
}
func doTransNorm(f Form, buf []byte, b []byte) []byte {
acc := []byte{}
for p := 0; p < len(b); {
nd, ns, _ := f.Transform(buf[:], b[p:], true)
p += ns
acc = append(acc, buf[:nd]...)
}
return acc
}
func TestTransformNorm(t *testing.T) {
for _, sz := range transBufSizes {
buf := make([]byte, sz)
runNormTests(t, fmt.Sprintf("Transform:%d", sz), func(f Form, out []byte, s string) []byte {
return doTransNorm(f, buf, append(out, s...))
})
}
}

View File

@@ -0,0 +1,232 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
type valueRange struct {
value uint16 // header: value:stride
lo, hi byte // header: lo:n
}
type trie struct {
index []uint8
values []uint16
sparse []valueRange
sparseOffset []uint16
cutoff uint8 // indices >= cutoff are sparse
}
// lookupValue determines the type of block n and looks up the value for b.
// For n < t.cutoff, the block is a simple lookup table. Otherwise, the block
// is a list of ranges with an accompanying value. Given a matching range r,
// the value for b is by r.value + (b - r.lo) * stride.
func (t *trie) lookupValue(n uint8, b byte) uint16 {
if n < t.cutoff {
return t.values[uint16(n)<<6+uint16(b)]
}
offset := t.sparseOffset[n-t.cutoff]
header := t.sparse[offset]
lo := offset + 1
hi := lo + uint16(header.lo)
for lo < hi {
m := lo + (hi-lo)/2
r := t.sparse[m]
if r.lo <= b && b <= r.hi {
return r.value + uint16(b-r.lo)*header.value
}
if b < r.lo {
hi = m
} else {
lo = m + 1
}
}
return 0
}
const (
t1 = 0x00 // 0000 0000
tx = 0x80 // 1000 0000
t2 = 0xC0 // 1100 0000
t3 = 0xE0 // 1110 0000
t4 = 0xF0 // 1111 0000
t5 = 0xF8 // 1111 1000
t6 = 0xFC // 1111 1100
te = 0xFE // 1111 1110
)
// lookup returns the trie value for the first UTF-8 encoding in s and
// the width in bytes of this encoding. The size will be 0 if s does not
// hold enough bytes to complete the encoding. len(s) must be greater than 0.
func (t *trie) lookup(s []byte) (v uint16, sz int) {
c0 := s[0]
switch {
case c0 < tx:
return t.values[c0], 1
case c0 < t2:
return 0, 1
case c0 < t3:
if len(s) < 2 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
return t.lookupValue(i, c1), 2
case c0 < t4:
if len(s) < 3 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
o := uint16(i)<<6 + uint16(c1)
i = t.index[o]
c2 := s[2]
if c2 < tx || t2 <= c2 {
return 0, 2
}
return t.lookupValue(i, c2), 3
case c0 < t5:
if len(s) < 4 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
o := uint16(i)<<6 + uint16(c1)
i = t.index[o]
c2 := s[2]
if c2 < tx || t2 <= c2 {
return 0, 2
}
o = uint16(i)<<6 + uint16(c2)
i = t.index[o]
c3 := s[3]
if c3 < tx || t2 <= c3 {
return 0, 3
}
return t.lookupValue(i, c3), 4
}
// Illegal rune
return 0, 1
}
// lookupString returns the trie value for the first UTF-8 encoding in s and
// the width in bytes of this encoding. The size will be 0 if s does not
// hold enough bytes to complete the encoding. len(s) must be greater than 0.
func (t *trie) lookupString(s string) (v uint16, sz int) {
c0 := s[0]
switch {
case c0 < tx:
return t.values[c0], 1
case c0 < t2:
return 0, 1
case c0 < t3:
if len(s) < 2 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
return t.lookupValue(i, c1), 2
case c0 < t4:
if len(s) < 3 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
o := uint16(i)<<6 + uint16(c1)
i = t.index[o]
c2 := s[2]
if c2 < tx || t2 <= c2 {
return 0, 2
}
return t.lookupValue(i, c2), 3
case c0 < t5:
if len(s) < 4 {
return 0, 0
}
i := t.index[c0]
c1 := s[1]
if c1 < tx || t2 <= c1 {
return 0, 1
}
o := uint16(i)<<6 + uint16(c1)
i = t.index[o]
c2 := s[2]
if c2 < tx || t2 <= c2 {
return 0, 2
}
o = uint16(i)<<6 + uint16(c2)
i = t.index[o]
c3 := s[3]
if c3 < tx || t2 <= c3 {
return 0, 3
}
return t.lookupValue(i, c3), 4
}
// Illegal rune
return 0, 1
}
// lookupUnsafe returns the trie value for the first UTF-8 encoding in s.
// s must hold a full encoding.
func (t *trie) lookupUnsafe(s []byte) uint16 {
c0 := s[0]
if c0 < tx {
return t.values[c0]
}
if c0 < t2 {
return 0
}
i := t.index[c0]
if c0 < t3 {
return t.lookupValue(i, s[1])
}
i = t.index[uint16(i)<<6+uint16(s[1])]
if c0 < t4 {
return t.lookupValue(i, s[2])
}
i = t.index[uint16(i)<<6+uint16(s[2])]
if c0 < t5 {
return t.lookupValue(i, s[3])
}
return 0
}
// lookupStringUnsafe returns the trie value for the first UTF-8 encoding in s.
// s must hold a full encoding.
func (t *trie) lookupStringUnsafe(s string) uint16 {
c0 := s[0]
if c0 < tx {
return t.values[c0]
}
if c0 < t2 {
return 0
}
i := t.index[c0]
if c0 < t3 {
return t.lookupValue(i, s[1])
}
i = t.index[uint16(i)<<6+uint16(s[1])]
if c0 < t4 {
return t.lookupValue(i, s[2])
}
i = t.index[uint16(i)<<6+uint16(s[2])]
if c0 < t5 {
return t.lookupValue(i, s[3])
}
return 0
}

View File

@@ -0,0 +1,152 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package norm
import (
"testing"
"unicode/utf8"
)
// Test data is located in triedata_test.go; generated by maketesttables.
var testdata = testdataTrie
type rangeTest struct {
block uint8
lookup byte
result uint16
table []valueRange
offsets []uint16
}
var range1Off = []uint16{0, 2}
var range1 = []valueRange{
{0, 1, 0},
{1, 0x80, 0x80},
{0, 2, 0},
{1, 0x80, 0x80},
{9, 0xff, 0xff},
}
var rangeTests = []rangeTest{
{10, 0x80, 1, range1, range1Off},
{10, 0x00, 0, range1, range1Off},
{11, 0x80, 1, range1, range1Off},
{11, 0xff, 9, range1, range1Off},
{11, 0x00, 0, range1, range1Off},
}
func TestLookupSparse(t *testing.T) {
for i, test := range rangeTests {
n := trie{sparse: test.table, sparseOffset: test.offsets, cutoff: 10}
v := n.lookupValue(test.block, test.lookup)
if v != test.result {
t.Errorf("LookupSparse:%d: found %X; want %X", i, v, test.result)
}
}
}
// Test cases for illegal runes.
type trietest struct {
size int
bytes []byte
}
var tests = []trietest{
// illegal runes
{1, []byte{0x80}},
{1, []byte{0xFF}},
{1, []byte{t2, tx - 1}},
{1, []byte{t2, t2}},
{2, []byte{t3, tx, tx - 1}},
{2, []byte{t3, tx, t2}},
{1, []byte{t3, tx - 1, tx}},
{3, []byte{t4, tx, tx, tx - 1}},
{3, []byte{t4, tx, tx, t2}},
{1, []byte{t4, t2, tx, tx - 1}},
{2, []byte{t4, tx, t2, tx - 1}},
// short runes
{0, []byte{t2}},
{0, []byte{t3, tx}},
{0, []byte{t4, tx, tx}},
// we only support UTF-8 up to utf8.UTFMax bytes (4 bytes)
{1, []byte{t5, tx, tx, tx, tx}},
{1, []byte{t6, tx, tx, tx, tx, tx}},
}
func mkUTF8(r rune) ([]byte, int) {
var b [utf8.UTFMax]byte
sz := utf8.EncodeRune(b[:], r)
return b[:sz], sz
}
func TestLookup(t *testing.T) {
for i, tt := range testRunes {
b, szg := mkUTF8(tt)
v, szt := testdata.lookup(b)
if int(v) != i {
t.Errorf("lookup(%U): found value %#x, expected %#x", tt, v, i)
}
if szt != szg {
t.Errorf("lookup(%U): found size %d, expected %d", tt, szt, szg)
}
}
for i, tt := range tests {
v, sz := testdata.lookup(tt.bytes)
if v != 0 {
t.Errorf("lookup of illegal rune, case %d: found value %#x, expected 0", i, v)
}
if sz != tt.size {
t.Errorf("lookup of illegal rune, case %d: found size %d, expected %d", i, sz, tt.size)
}
}
// Verify defaults.
if v, _ := testdata.lookup([]byte{0xC1, 0x8C}); v != 0 {
t.Errorf("lookup of non-existing rune should be 0; found %X", v)
}
}
func TestLookupUnsafe(t *testing.T) {
for i, tt := range testRunes {
b, _ := mkUTF8(tt)
v := testdata.lookupUnsafe(b)
if int(v) != i {
t.Errorf("lookupUnsafe(%U): found value %#x, expected %#x", i, v, i)
}
}
}
func TestLookupString(t *testing.T) {
for i, tt := range testRunes {
b, szg := mkUTF8(tt)
v, szt := testdata.lookupString(string(b))
if int(v) != i {
t.Errorf("lookup(%U): found value %#x, expected %#x", i, v, i)
}
if szt != szg {
t.Errorf("lookup(%U): found size %d, expected %d", i, szt, szg)
}
}
for i, tt := range tests {
v, sz := testdata.lookupString(string(tt.bytes))
if int(v) != 0 {
t.Errorf("lookup of illegal rune, case %d: found value %#x, expected 0", i, v)
}
if sz != tt.size {
t.Errorf("lookup of illegal rune, case %d: found size %d, expected %d", i, sz, tt.size)
}
}
}
func TestLookupStringUnsafe(t *testing.T) {
for i, tt := range testRunes {
b, _ := mkUTF8(tt)
v := testdata.lookupStringUnsafe(string(b))
if int(v) != i {
t.Errorf("lookupUnsafe(%U): found value %#x, expected %#x", i, v, i)
}
}
}

View File

@@ -0,0 +1,85 @@
// Generated by running
// maketesttables
// DO NOT EDIT
package norm
var testRunes = []int32{1, 12, 127, 128, 256, 2047, 2048, 2457, 65535, 65536, 65793, 1114111, 512, 513, 514, 528, 533}
// testdataValues: 192 entries, 384 bytes
// Block 2 is the null block.
var testdataValues = [192]uint16{
// Block 0x0, offset 0x0
0x000c: 0x0001,
// Block 0x1, offset 0x40
0x007f: 0x0002,
// Block 0x2, offset 0x80
}
// testdataSparseOffset: 10 entries, 20 bytes
var testdataSparseOffset = []uint16{0x0, 0x2, 0x4, 0x8, 0xa, 0xc, 0xe, 0x10, 0x12, 0x14}
// testdataSparseValues: 22 entries, 88 bytes
var testdataSparseValues = [22]valueRange{
// Block 0x0, offset 0x1
{value: 0x0000, lo: 0x01},
{value: 0x0003, lo: 0x80, hi: 0x80},
// Block 0x1, offset 0x2
{value: 0x0000, lo: 0x01},
{value: 0x0004, lo: 0x80, hi: 0x80},
// Block 0x2, offset 0x3
{value: 0x0001, lo: 0x03},
{value: 0x000c, lo: 0x80, hi: 0x82},
{value: 0x000f, lo: 0x90, hi: 0x90},
{value: 0x0010, lo: 0x95, hi: 0x95},
// Block 0x3, offset 0x4
{value: 0x0000, lo: 0x01},
{value: 0x0005, lo: 0xbf, hi: 0xbf},
// Block 0x4, offset 0x5
{value: 0x0000, lo: 0x01},
{value: 0x0006, lo: 0x80, hi: 0x80},
// Block 0x5, offset 0x6
{value: 0x0000, lo: 0x01},
{value: 0x0007, lo: 0x99, hi: 0x99},
// Block 0x6, offset 0x7
{value: 0x0000, lo: 0x01},
{value: 0x0008, lo: 0xbf, hi: 0xbf},
// Block 0x7, offset 0x8
{value: 0x0000, lo: 0x01},
{value: 0x0009, lo: 0x80, hi: 0x80},
// Block 0x8, offset 0x9
{value: 0x0000, lo: 0x01},
{value: 0x000a, lo: 0x81, hi: 0x81},
// Block 0x9, offset 0xa
{value: 0x0000, lo: 0x01},
{value: 0x000b, lo: 0xbf, hi: 0xbf},
}
// testdataLookup: 640 bytes
// Block 0 is the null block.
var testdataLookup = [640]uint8{
// Block 0x0, offset 0x0
// Block 0x1, offset 0x40
// Block 0x2, offset 0x80
// Block 0x3, offset 0xc0
0x0c2: 0x01, 0x0c4: 0x02,
0x0c8: 0x03,
0x0df: 0x04,
0x0e0: 0x02,
0x0ef: 0x03,
0x0f0: 0x05, 0x0f4: 0x07,
// Block 0x4, offset 0x100
0x120: 0x05, 0x126: 0x06,
// Block 0x5, offset 0x140
0x17f: 0x07,
// Block 0x6, offset 0x180
0x180: 0x08, 0x184: 0x09,
// Block 0x7, offset 0x1c0
0x1d0: 0x04,
// Block 0x8, offset 0x200
0x23f: 0x0a,
// Block 0x9, offset 0x240
0x24f: 0x06,
}
var testdataTrie = trie{testdataLookup[:], testdataValues[:], testdataSparseValues[:], testdataSparseOffset[:], 1}

View File

@@ -0,0 +1,317 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// +build ignore
// Trie table generator.
// Used by make*tables tools to generate a go file with trie data structures
// for mapping UTF-8 to a 16-bit value. All but the last byte in a UTF-8 byte
// sequence are used to lookup offsets in the index table to be used for the
// next byte. The last byte is used to index into a table with 16-bit values.
package main
import (
"fmt"
"hash/crc32"
"log"
"unicode/utf8"
)
const (
blockSize = 64
blockOffset = 2 // Subtract two blocks to compensate for the 0x80 added to continuation bytes.
maxSparseEntries = 16
)
// Intermediate trie structure
type trieNode struct {
table [256]*trieNode
value int
b byte
leaf bool
}
func newNode() *trieNode {
return new(trieNode)
}
func (n trieNode) String() string {
s := fmt.Sprint("trieNode{table: { non-nil at index: ")
for i, v := range n.table {
if v != nil {
s += fmt.Sprintf("%d, ", i)
}
}
s += fmt.Sprintf("}, value:%#x, b:%#x leaf:%v}", n.value, n.b, n.leaf)
return s
}
func (n trieNode) isInternal() bool {
internal := true
for i := 0; i < 256; i++ {
if nn := n.table[i]; nn != nil {
if !internal && !nn.leaf {
log.Fatalf("triegen: isInternal: node contains both leaf and non-leaf children (%v)", n)
}
internal = internal && !nn.leaf
}
}
return internal
}
func (n trieNode) mostFrequentStride() int {
counts := make(map[int]int)
v := 0
for _, t := range n.table[0x80 : 0x80+blockSize] {
if t != nil {
if stride := t.value - v; v != 0 && stride >= 0 {
counts[stride]++
}
v = t.value
} else {
v = 0
}
}
var maxs, maxc int
for stride, cnt := range counts {
if cnt > maxc || (cnt == maxc && stride < maxs) {
maxs, maxc = stride, cnt
}
}
return maxs
}
func (n trieNode) countSparseEntries() int {
stride := n.mostFrequentStride()
var count, v int
for _, t := range n.table[0x80 : 0x80+blockSize] {
tv := 0
if t != nil {
tv = t.value
}
if tv-v != stride {
if tv != 0 {
count++
}
}
v = tv
}
return count
}
func (n *trieNode) insert(r rune, value uint16) {
var p [utf8.UTFMax]byte
sz := utf8.EncodeRune(p[:], r)
for i := 0; i < sz; i++ {
if n.leaf {
log.Fatalf("triegen: insert: node (%#v) should not be a leaf", n)
}
nn := n.table[p[i]]
if nn == nil {
nn = newNode()
nn.b = p[i]
n.table[p[i]] = nn
}
n = nn
}
n.value = int(value)
n.leaf = true
}
type nodeIndex struct {
lookupBlocks []*trieNode
valueBlocks []*trieNode
sparseBlocks []*trieNode
sparseOffset []uint16
sparseCount int
lookupBlockIdx map[uint32]int
valueBlockIdx map[uint32]int
}
func newIndex() *nodeIndex {
index := &nodeIndex{}
index.lookupBlocks = make([]*trieNode, 0)
index.valueBlocks = make([]*trieNode, 0)
index.sparseBlocks = make([]*trieNode, 0)
index.sparseOffset = make([]uint16, 1)
index.lookupBlockIdx = make(map[uint32]int)
index.valueBlockIdx = make(map[uint32]int)
return index
}
func computeOffsets(index *nodeIndex, n *trieNode) int {
if n.leaf {
return n.value
}
hasher := crc32.New(crc32.MakeTable(crc32.IEEE))
// We only index continuation bytes.
for i := 0; i < blockSize; i++ {
v := 0
if nn := n.table[0x80+i]; nn != nil {
v = computeOffsets(index, nn)
}
hasher.Write([]byte{uint8(v >> 8), uint8(v)})
}
h := hasher.Sum32()
if n.isInternal() {
v, ok := index.lookupBlockIdx[h]
if !ok {
v = len(index.lookupBlocks) - blockOffset
index.lookupBlocks = append(index.lookupBlocks, n)
index.lookupBlockIdx[h] = v
}
n.value = v
} else {
v, ok := index.valueBlockIdx[h]
if !ok {
if c := n.countSparseEntries(); c > maxSparseEntries {
v = len(index.valueBlocks) - blockOffset
index.valueBlocks = append(index.valueBlocks, n)
index.valueBlockIdx[h] = v
} else {
v = -len(index.sparseOffset)
index.sparseBlocks = append(index.sparseBlocks, n)
index.sparseOffset = append(index.sparseOffset, uint16(index.sparseCount))
index.sparseCount += c + 1
index.valueBlockIdx[h] = v
}
}
n.value = v
}
return n.value
}
func printValueBlock(nr int, n *trieNode, offset int) {
boff := nr * blockSize
fmt.Printf("\n// Block %#x, offset %#x", nr, boff)
var printnewline bool
for i := 0; i < blockSize; i++ {
if i%6 == 0 {
printnewline = true
}
v := 0
if nn := n.table[i+offset]; nn != nil {
v = nn.value
}
if v != 0 {
if printnewline {
fmt.Printf("\n")
printnewline = false
}
fmt.Printf("%#04x:%#04x, ", boff+i, v)
}
}
}
func printSparseBlock(nr int, n *trieNode) {
boff := -n.value
fmt.Printf("\n// Block %#x, offset %#x", nr, boff)
v := 0
//stride := f(n)
stride := n.mostFrequentStride()
c := n.countSparseEntries()
fmt.Printf("\n{value:%#04x,lo:%#02x},", stride, uint8(c))
for i, nn := range n.table[0x80 : 0x80+blockSize] {
nv := 0
if nn != nil {
nv = nn.value
}
if nv-v != stride {
if v != 0 {
fmt.Printf(",hi:%#02x},", 0x80+i-1)
}
if nv != 0 {
fmt.Printf("\n{value:%#04x,lo:%#02x", nv, nn.b)
}
}
v = nv
}
if v != 0 {
fmt.Printf(",hi:%#02x},", 0x80+blockSize-1)
}
}
func printLookupBlock(nr int, n *trieNode, offset, cutoff int) {
boff := nr * blockSize
fmt.Printf("\n// Block %#x, offset %#x", nr, boff)
var printnewline bool
for i := 0; i < blockSize; i++ {
if i%8 == 0 {
printnewline = true
}
v := 0
if nn := n.table[i+offset]; nn != nil {
v = nn.value
}
if v != 0 {
if v < 0 {
v = -v - 1 + cutoff
}
if printnewline {
fmt.Printf("\n")
printnewline = false
}
fmt.Printf("%#03x:%#02x, ", boff+i, v)
}
}
}
// printTables returns the size in bytes of the generated tables.
func (t *trieNode) printTables(name string) int {
index := newIndex()
// Values for 7-bit ASCII are stored in first two block, followed by nil block.
index.valueBlocks = append(index.valueBlocks, nil, nil, nil)
// First byte of multi-byte UTF-8 codepoints are indexed in 4th block.
index.lookupBlocks = append(index.lookupBlocks, nil, nil, nil, nil)
// Index starter bytes of multi-byte UTF-8.
for i := 0xC0; i < 0x100; i++ {
if t.table[i] != nil {
computeOffsets(index, t.table[i])
}
}
nv := len(index.valueBlocks) * blockSize
fmt.Printf("// %sValues: %d entries, %d bytes\n", name, nv, nv*2)
fmt.Printf("// Block 2 is the null block.\n")
fmt.Printf("var %sValues = [%d]uint16 {", name, nv)
printValueBlock(0, t, 0)
printValueBlock(1, t, 64)
printValueBlock(2, newNode(), 0)
for i := 3; i < len(index.valueBlocks); i++ {
printValueBlock(i, index.valueBlocks[i], 0x80)
}
fmt.Print("\n}\n\n")
ls := len(index.sparseBlocks)
fmt.Printf("// %sSparseOffset: %d entries, %d bytes\n", name, ls, ls*2)
fmt.Printf("var %sSparseOffset = %#v\n\n", name, index.sparseOffset[1:])
ns := index.sparseCount
fmt.Printf("// %sSparseValues: %d entries, %d bytes\n", name, ns, ns*4)
fmt.Printf("var %sSparseValues = [%d]valueRange {", name, ns)
for i, n := range index.sparseBlocks {
printSparseBlock(i, n)
}
fmt.Print("\n}\n\n")
cutoff := len(index.valueBlocks) - blockOffset
ni := len(index.lookupBlocks) * blockSize
fmt.Printf("// %sLookup: %d bytes\n", name, ni)
fmt.Printf("// Block 0 is the null block.\n")
fmt.Printf("var %sLookup = [%d]uint8 {", name, ni)
printLookupBlock(0, newNode(), 0, cutoff)
printLookupBlock(1, newNode(), 0, cutoff)
printLookupBlock(2, newNode(), 0, cutoff)
printLookupBlock(3, t, 0xC0, cutoff)
for i := 4; i < len(index.lookupBlocks); i++ {
printLookupBlock(i, index.lookupBlocks[i], 0x80, cutoff)
}
fmt.Print("\n}\n\n")
fmt.Printf("var %sTrie = trie{ %sLookup[:], %sValues[:], %sSparseValues[:], %sSparseOffset[:], %d}\n\n",
name, name, name, name, name, cutoff)
return nv*2 + ns*4 + ni + ls*2
}

View File

@@ -0,0 +1,2 @@
inject
inject.test

View File

@@ -0,0 +1,20 @@
The MIT License (MIT)
Copyright (c) 2013 Jeremy Saenz
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -0,0 +1,4 @@
inject
======
Dependency injection for go

View File

@@ -0,0 +1,168 @@
// Package inject provides utilities for mapping and injecting dependencies in various ways.
package inject
import (
"fmt"
"reflect"
)
// Injector represents an interface for mapping and injecting dependencies into structs
// and function arguments.
type Injector interface {
Applicator
Invoker
TypeMapper
// SetParent sets the parent of the injector. If the injector cannot find a
// dependency in its Type map it will check its parent before returning an
// error.
SetParent(Injector)
}
// Applicator represents an interface for mapping dependencies to a struct.
type Applicator interface {
// Maps dependencies in the Type map to each field in the struct
// that is tagged with 'inject'. Returns an error if the injection
// fails.
Apply(interface{}) error
}
// Invoker represents an interface for calling functions via reflection.
type Invoker interface {
// Invoke attempts to call the interface{} provided as a function,
// providing dependencies for function arguments based on Type. Returns
// a slice of reflect.Value representing the returned values of the function.
// Returns an error if the injection fails.
Invoke(interface{}) ([]reflect.Value, error)
}
// TypeMapper represents an interface for mapping interface{} values based on type.
type TypeMapper interface {
// Maps the interface{} value based on its immediate type from reflect.TypeOf.
Map(interface{}) TypeMapper
// Maps the interface{} value based on the pointer of an Interface provided.
// This is really only useful for mapping a value as an interface, as interfaces
// cannot at this time be referenced directly without a pointer.
MapTo(interface{}, interface{}) TypeMapper
// Provides a possibility to directly insert a mapping based on type and value.
// This makes it possible to directly map type arguments not possible to instantiate
// with reflect like unidirectional channels.
Set(reflect.Type, reflect.Value) TypeMapper
// Returns the Value that is mapped to the current type. Returns a zeroed Value if
// the Type has not been mapped.
Get(reflect.Type) reflect.Value
}
type injector struct {
values map[reflect.Type]reflect.Value
parent Injector
}
// InterfaceOf dereferences a pointer to an Interface type.
// It panics if value is not an pointer to an interface.
func InterfaceOf(value interface{}) reflect.Type {
t := reflect.TypeOf(value)
for t.Kind() == reflect.Ptr {
t = t.Elem()
}
if t.Kind() != reflect.Interface {
panic("Called inject.InterfaceOf with a value that is not a pointer to an interface. (*MyInterface)(nil)")
}
return t
}
// New returns a new Injector.
func New() Injector {
return &injector{
values: make(map[reflect.Type]reflect.Value),
}
}
// Invoke attempts to call the interface{} provided as a function,
// providing dependencies for function arguments based on Type.
// Returns a slice of reflect.Value representing the returned values of the function.
// Returns an error if the injection fails.
// It panics if f is not a function
func (inj *injector) Invoke(f interface{}) ([]reflect.Value, error) {
t := reflect.TypeOf(f)
var in = make([]reflect.Value, t.NumIn()) //Panic if t is not kind of Func
for i := 0; i < t.NumIn(); i++ {
argType := t.In(i)
val := inj.Get(argType)
if !val.IsValid() {
return nil, fmt.Errorf("Value not found for type %v", argType)
}
in[i] = val
}
return reflect.ValueOf(f).Call(in), nil
}
// Maps dependencies in the Type map to each field in the struct
// that is tagged with 'inject'.
// Returns an error if the injection fails.
func (inj *injector) Apply(val interface{}) error {
v := reflect.ValueOf(val)
for v.Kind() == reflect.Ptr {
v = v.Elem()
}
if v.Kind() != reflect.Struct {
return nil // Should not panic here ?
}
t := v.Type()
for i := 0; i < v.NumField(); i++ {
f := v.Field(i)
structField := t.Field(i)
if f.CanSet() && structField.Tag == "inject" {
ft := f.Type()
v := inj.Get(ft)
if !v.IsValid() {
return fmt.Errorf("Value not found for type %v", ft)
}
f.Set(v)
}
}
return nil
}
// Maps the concrete value of val to its dynamic type using reflect.TypeOf,
// It returns the TypeMapper registered in.
func (i *injector) Map(val interface{}) TypeMapper {
i.values[reflect.TypeOf(val)] = reflect.ValueOf(val)
return i
}
func (i *injector) MapTo(val interface{}, ifacePtr interface{}) TypeMapper {
i.values[InterfaceOf(ifacePtr)] = reflect.ValueOf(val)
return i
}
// Maps the given reflect.Type to the given reflect.Value and returns
// the Typemapper the mapping has been registered in.
func (i *injector) Set(typ reflect.Type, val reflect.Value) TypeMapper {
i.values[typ] = val
return i
}
func (i *injector) Get(t reflect.Type) reflect.Value {
val := i.values[t]
if !val.IsValid() && i.parent != nil {
val = i.parent.Get(t)
}
return val
}
func (i *injector) SetParent(parent Injector) {
i.parent = parent
}

View File

@@ -0,0 +1,142 @@
package inject_test
import (
"github.com/codegangsta/inject"
"reflect"
"testing"
)
type SpecialString interface {
}
type TestStruct struct {
Dep1 string `inject`
Dep2 SpecialString `inject`
Dep3 string
}
/* Test Helpers */
func expect(t *testing.T, a interface{}, b interface{}) {
if a != b {
t.Errorf("Expected %v (type %v) - Got %v (type %v)", b, reflect.TypeOf(b), a, reflect.TypeOf(a))
}
}
func refute(t *testing.T, a interface{}, b interface{}) {
if a == b {
t.Errorf("Did not expect %v (type %v) - Got %v (type %v)", b, reflect.TypeOf(b), a, reflect.TypeOf(a))
}
}
func Test_InjectorInvoke(t *testing.T) {
injector := inject.New()
expect(t, injector == nil, false)
dep := "some dependency"
injector.Map(dep)
dep2 := "another dep"
injector.MapTo(dep2, (*SpecialString)(nil))
dep3 := make(chan *SpecialString)
dep4 := make(chan *SpecialString)
typRecv := reflect.ChanOf(reflect.RecvDir, reflect.TypeOf(dep3).Elem())
typSend := reflect.ChanOf(reflect.SendDir, reflect.TypeOf(dep4).Elem())
injector.Set(typRecv, reflect.ValueOf(dep3))
injector.Set(typSend, reflect.ValueOf(dep4))
_, err := injector.Invoke(func(d1 string, d2 SpecialString, d3 <-chan *SpecialString, d4 chan<- *SpecialString) {
expect(t, d1, dep)
expect(t, d2, dep2)
expect(t, reflect.TypeOf(d3).Elem(), reflect.TypeOf(dep3).Elem())
expect(t, reflect.TypeOf(d4).Elem(), reflect.TypeOf(dep4).Elem())
expect(t, reflect.TypeOf(d3).ChanDir(), reflect.RecvDir)
expect(t, reflect.TypeOf(d4).ChanDir(), reflect.SendDir)
})
expect(t, err, nil)
}
func Test_InjectorInvokeReturnValues(t *testing.T) {
injector := inject.New()
expect(t, injector == nil, false)
dep := "some dependency"
injector.Map(dep)
dep2 := "another dep"
injector.MapTo(dep2, (*SpecialString)(nil))
result, err := injector.Invoke(func(d1 string, d2 SpecialString) string {
expect(t, d1, dep)
expect(t, d2, dep2)
return "Hello world"
})
expect(t, result[0].String(), "Hello world")
expect(t, err, nil)
}
func Test_InjectorApply(t *testing.T) {
injector := inject.New()
injector.Map("a dep").MapTo("another dep", (*SpecialString)(nil))
s := TestStruct{}
err := injector.Apply(&s)
expect(t, err, nil)
expect(t, s.Dep1, "a dep")
expect(t, s.Dep2, "another dep")
}
func Test_InterfaceOf(t *testing.T) {
iType := inject.InterfaceOf((*SpecialString)(nil))
expect(t, iType.Kind(), reflect.Interface)
iType = inject.InterfaceOf((**SpecialString)(nil))
expect(t, iType.Kind(), reflect.Interface)
// Expecting nil
defer func() {
rec := recover()
refute(t, rec, nil)
}()
iType = inject.InterfaceOf((*testing.T)(nil))
}
func Test_InjectorSet(t *testing.T) {
injector := inject.New()
typ := reflect.TypeOf("string")
typSend := reflect.ChanOf(reflect.SendDir, typ)
typRecv := reflect.ChanOf(reflect.RecvDir, typ)
// instantiating unidirectional channels is not possible using reflect
// http://golang.org/src/pkg/reflect/value.go?s=60463:60504#L2064
chanRecv := reflect.MakeChan(reflect.ChanOf(reflect.BothDir, typ), 0)
chanSend := reflect.MakeChan(reflect.ChanOf(reflect.BothDir, typ), 0)
injector.Set(typSend, chanSend)
injector.Set(typRecv, chanRecv)
expect(t, injector.Get(typSend).IsValid(), true)
expect(t, injector.Get(typRecv).IsValid(), true)
expect(t, injector.Get(chanSend.Type()).IsValid(), false)
}
func Test_InjectorGet(t *testing.T) {
injector := inject.New()
injector.Map("some dependency")
expect(t, injector.Get(reflect.TypeOf("string")).IsValid(), true)
expect(t, injector.Get(reflect.TypeOf(11)).IsValid(), false)
}
func Test_InjectorSetParent(t *testing.T) {
injector := inject.New()
injector.MapTo("another dep", (*SpecialString)(nil))
injector2 := inject.New()
injector2.SetParent(injector)
expect(t, injector2.Get(inject.InterfaceOf((*SpecialString)(nil))).IsValid(), true)
}

View File

@@ -0,0 +1,23 @@
# Compiled Object files, Static and Dynamic libs (Shared Objects)
*.o
*.a
*.so
# Folders
_obj
_test
# Architecture specific extensions/prefixes
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.test

View File

@@ -0,0 +1,20 @@
The MIT License (MIT)
Copyright (c) 2013 Jeremy Saenz
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

@@ -0,0 +1,345 @@
# Martini [![wercker status](https://app.wercker.com/status/174bef7e3c999e103cacfe2770102266 "wercker status")](https://app.wercker.com/project/bykey/174bef7e3c999e103cacfe2770102266) [![GoDoc](https://godoc.org/github.com/codegangsta/martini?status.png)](http://godoc.org/github.com/codegangsta/martini)
Martini is a powerful package for quickly writing modular web applications/services in Golang.
Language Translations: [Simplified Chinese (zh_CN)](translations/README_zh_cn.md)
## Getting Started
After installing Go and setting up your [GOPATH](http://golang.org/doc/code.html#GOPATH), create your first `.go` file. We'll call it `server.go`.
~~~ go
package main
import "github.com/codegangsta/martini"
func main() {
m := martini.Classic()
m.Get("/", func() string {
return "Hello world!"
})
m.Run()
}
~~~
Then install the Martini package (**go 1.1** and greater is required):
~~~
go get github.com/codegangsta/martini
~~~
Then run your server:
~~~
go run server.go
~~~
You will now have a Martini webserver running on `localhost:3000`.
## Getting Help
Join the [Mailing list](https://groups.google.com/forum/#!forum/martini-go)
Watch the [Demo Video](http://martini.codegangsta.io/#demo)
## Features
* Extremely simple to use.
* Non-intrusive design.
* Plays nice with other Golang packages.
* Awesome path matching and routing.
* Modular design - Easy to add functionality, easy to rip stuff out.
* Lots of good handlers/middlewares to use.
* Great 'out of the box' feature set.
* **Fully compatible with the [http.HandlerFunc](http://godoc.org/net/http#HandlerFunc) interface.**
## More Middleware
For more middleware and functionality, check out the repositories in the [martini-contrib](https://github.com/martini-contrib) organization.
## Table of Contents
* [Classic Martini](#classic-martini)
* [Handlers](#handlers)
* [Routing](#routing)
* [Services](#services)
* [Serving Static Files](#serving-static-files)
* [Middleware Handlers](#middleware-handlers)
* [Next()](#next)
* [Martini Env](#martini-env)
* [FAQ](#faq)
## Classic Martini
To get up and running quickly, [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic) provides some reasonable defaults that work well for most web applications:
~~~ go
m := martini.Classic()
// ... middleware and routing goes here
m.Run()
~~~
Below is some of the functionality [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic) pulls in automatically:
* Request/Response Logging - [martini.Logger](http://godoc.org/github.com/codegangsta/martini#Logger)
* Panic Recovery - [martini.Recovery](http://godoc.org/github.com/codegangsta/martini#Recovery)
* Static File serving - [martini.Static](http://godoc.org/github.com/codegangsta/martini#Static)
* Routing - [martini.Router](http://godoc.org/github.com/codegangsta/martini#Router)
### Handlers
Handlers are the heart and soul of Martini. A handler is basically any kind of callable function:
~~~ go
m.Get("/", func() {
println("hello world")
})
~~~
#### Return Values
If a handler returns something, Martini will write the result to the current [http.ResponseWriter](http://godoc.org/net/http#ResponseWriter) as a string:
~~~ go
m.Get("/", func() string {
return "hello world" // HTTP 200 : "hello world"
})
~~~
You can also optionally return a status code:
~~~ go
m.Get("/", func() (int, string) {
return 418, "i'm a teapot" // HTTP 418 : "i'm a teapot"
})
~~~
#### Service Injection
Handlers are invoked via reflection. Martini makes use of *Dependency Injection* to resolve dependencies in a Handlers argument list. **This makes Martini completely compatible with golang's `http.HandlerFunc` interface.**
If you add an argument to your Handler, Martini will search its list of services and attempt to resolve the dependency via type assertion:
~~~ go
m.Get("/", func(res http.ResponseWriter, req *http.Request) { // res and req are injected by Martini
res.WriteHeader(200) // HTTP 200
})
~~~
The following services are included with [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic):
* [*log.Logger](http://godoc.org/log#Logger) - Global logger for Martini.
* [martini.Context](http://godoc.org/github.com/codegangsta/martini#Context) - http request context.
* [martini.Params](http://godoc.org/github.com/codegangsta/martini#Params) - `map[string]string` of named params found by route matching.
* [martini.Routes](http://godoc.org/github.com/codegangsta/martini#Routes) - Route helper service.
* [http.ResponseWriter](http://godoc.org/net/http/#ResponseWriter) - http Response writer interface.
* [*http.Request](http://godoc.org/net/http/#Request) - http Request.
### Routing
In Martini, a route is an HTTP method paired with a URL-matching pattern.
Each route can take one or more handler methods:
~~~ go
m.Get("/", func() {
// show something
})
m.Patch("/", func() {
// update something
})
m.Post("/", func() {
// create something
})
m.Put("/", func() {
// replace something
})
m.Delete("/", func() {
// destroy something
})
m.Options("/", func() {
// http options
})
m.NotFound(func() {
// handle 404
})
~~~
Routes are matched in the order they are defined. The first route that
matches the request is invoked.
Route patterns may include named parameters, accessible via the [martini.Params](http://godoc.org/github.com/codegangsta/martini#Params) service:
~~~ go
m.Get("/hello/:name", func(params martini.Params) string {
return "Hello " + params["name"]
})
~~~
Routes can be matched with regular expressions and globs as well:
~~~ go
m.Get("/hello/**", func(params martini.Params) string {
return "Hello " + params["_1"]
})
~~~
Route handlers can be stacked on top of each other, which is useful for things like authentication and authorization:
~~~ go
m.Get("/secret", authorize, func() {
// this will execute as long as authorize doesn't write a response
})
~~~
Route groups can be added too using the Group method.
~~~ go
m.Group("/books", func(r Router) {
r.Get("/:id", GetBooks)
r.Post("/new", NewBook)
r.Put("/update/:id", UpdateBook)
r.Delete("/delete/:id", DeleteBook)
})
~~~
Just like you can pass middlewares to a handler you can pass middlewares to groups.
~~~ go
m.Group("/books", func(r Router) {
r.Get("/:id", GetBooks)
r.Post("/new", NewBook)
r.Put("/update/:id", UpdateBook)
r.Delete("/delete/:id", DeleteBook)
}, MyMiddleware1, MyMiddleware2)
~~~
### Services
Services are objects that are available to be injected into a Handler's argument list. You can map a service on a *Global* or *Request* level.
#### Global Mapping
A Martini instance implements the inject.Injector interface, so mapping a service is easy:
~~~ go
db := &MyDatabase{}
m := martini.Classic()
m.Map(db) // the service will be available to all handlers as *MyDatabase
// ...
m.Run()
~~~
#### Request-Level Mapping
Mapping on the request level can be done in a handler via [martini.Context](http://godoc.org/github.com/codegangsta/martini#Context):
~~~ go
func MyCustomLoggerHandler(c martini.Context, req *http.Request) {
logger := &MyCustomLogger{req}
c.Map(logger) // mapped as *MyCustomLogger
}
~~~
#### Mapping values to Interfaces
One of the most powerful parts about services is the ability to map a service to an interface. For instance, if you wanted to override the [http.ResponseWriter](http://godoc.org/net/http#ResponseWriter) with an object that wrapped it and performed extra operations, you can write the following handler:
~~~ go
func WrapResponseWriter(res http.ResponseWriter, c martini.Context) {
rw := NewSpecialResponseWriter(res)
c.MapTo(rw, (*http.ResponseWriter)(nil)) // override ResponseWriter with our wrapper ResponseWriter
}
~~~
### Serving Static Files
A [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic) instance automatically serves static files from the "public" directory in the root of your server.
You can serve from more directories by adding more [martini.Static](http://godoc.org/github.com/codegangsta/martini#Static) handlers.
~~~ go
m.Use(martini.Static("assets")) // serve from the "assets" directory as well
~~~
## Middleware Handlers
Middleware Handlers sit between the incoming http request and the router. In essence they are no different than any other Handler in Martini. You can add a middleware handler to the stack like so:
~~~ go
m.Use(func() {
// do some middleware stuff
})
~~~
You can have full control over the middleware stack with the `Handlers` function. This will replace any handlers that have been previously set:
~~~ go
m.Handlers(
Middleware1,
Middleware2,
Middleware3,
)
~~~
Middleware Handlers work really well for things like logging, authorization, authentication, sessions, gzipping, error pages and any other operations that must happen before or after an http request:
~~~ go
// validate an api key
m.Use(func(res http.ResponseWriter, req *http.Request) {
if req.Header.Get("X-API-KEY") != "secret123" {
res.WriteHeader(http.StatusUnauthorized)
}
})
~~~
### Next()
[Context.Next()](http://godoc.org/github.com/codegangsta/martini#Context) is an optional function that Middleware Handlers can call to yield the until after the other Handlers have been executed. This works really well for any operations that must happen after an http request:
~~~ go
// log before and after a request
m.Use(func(c martini.Context, log *log.Logger){
log.Println("before a request")
c.Next()
log.Println("after a request")
})
~~~
## Martini Env
Some Martini handlers make use of the `martini.Env` global variable to provide special functionality for development environments vs production environments. It is reccomended that the `MARTINI_ENV=production` environment variable to be set when deploying a Martini server into a production environment.
## FAQ
### Where do I find middleware X?
Start by looking in the [martini-contrib](https://github.com/martini-contrib) projects. If it is not there feel free to contact a martini-contrib team member about adding a new repo to the organization.
* [auth](https://github.com/martini-contrib/auth) - Handlers for authentication.
* [binding](https://github.com/martini-contrib/binding) - Handler for mapping/validating a raw request into a structure.
* [gzip](https://github.com/martini-contrib/gzip) - Handler for adding gzip compress to requests
* [render](https://github.com/martini-contrib/render) - Handler that provides a service for easily rendering JSON and HTML templates.
* [acceptlang](https://github.com/martini-contrib/acceptlang) - Handler for parsing the `Accept-Language` HTTP header.
* [sessions](https://github.com/martini-contrib/sessions) - Handler that provides a Session service.
* [strip](https://github.com/martini-contrib/strip) - URL Prefix stripping.
* [method](https://github.com/martini-contrib/method) - HTTP method overriding via Header or form fields.
* [secure](https://github.com/martini-contrib/secure) - Implements a few quick security wins.
* [encoder](https://github.com/martini-contrib/encoder) - Encoder service for rendering data in several formats and content negotiation.
* [cors](https://github.com/martini-contrib/cors) - Handler that enables CORS support.
* [oauth2](https://github.com/martini-contrib/oauth2) - Handler that provides OAuth 2.0 login for Martini apps. Google Sign-in, Facebook Connect and Github login is supported.
### How do I integrate with existing servers?
A Martini instance implements `http.Handler`, so it can easily be used to serve subtrees
on existing Go servers. For example this is a working Martini app for Google App Engine:
~~~ go
package hello
import (
"net/http"
"github.com/codegangsta/martini"
)
func init() {
m := martini.Classic()
m.Get("/", func() string {
return "Hello world!"
})
http.Handle("/", m)
}
~~~
### How do I change the port/host?
Martini's `Run` function looks for the PORT and HOST environment variables and uses those. Otherwise Martini will default to localhost:3000.
To have more flexibility over port and host, use the `http.ListenAndServe` function instead.
~~~ go
m := martini.Classic()
// ...
log.Fatal(http.ListenAndServe(":8080", m))
~~~
### Live code reload?
[gin](https://github.com/codegangsta/gin) and [fresh](https://github.com/pilu/fresh) both live reload martini apps.
## Contributing
Martini is meant to be kept tiny and clean. Most contributions should end up in a repository in the [martini-contrib](https://github.com/martini-contrib) organization. If you do have a contribution for the core of Martini feel free to put up a Pull Request.
## About
Inspired by [express](https://github.com/visionmedia/express) and [sinatra](https://github.com/sinatra/sinatra)
Martini is obsessively designed by none other than the [Code Gangsta](http://codegangsta.io/)

View File

@@ -0,0 +1,25 @@
package martini
import (
"os"
)
// Envs
const (
Dev string = "development"
Prod string = "production"
Test string = "test"
)
// Env is the environment that Martini is executing in. The MARTINI_ENV is read on initialization to set this variable.
var Env = Dev
func setENV(e string) {
if len(e) > 0 {
Env = e
}
}
func init() {
setENV(os.Getenv("MARTINI_ENV"))
}

View File

@@ -0,0 +1,22 @@
package martini
import (
"testing"
)
func Test_SetENV(t *testing.T) {
tests := []struct {
in string
out string
}{
{"", "development"},
{"not_development", "not_development"},
}
for _, test := range tests {
setENV(test.in)
if Env != test.out {
expect(t, Env, test.out)
}
}
}

View File

@@ -0,0 +1,7 @@
// +build !go1.1
package martini
func MartiniDoesNotSupportGo1Point0() {
"Martini requires Go 1.1 or greater."
}

View File

@@ -0,0 +1,20 @@
package martini
import (
"log"
"net/http"
"time"
)
// Logger returns a middleware handler that logs the request as it goes in and the response as it goes out.
func Logger() Handler {
return func(res http.ResponseWriter, req *http.Request, c Context, log *log.Logger) {
start := time.Now()
log.Printf("Started %s %s", req.Method, req.URL.Path)
rw := res.(ResponseWriter)
c.Next()
log.Printf("Completed %v %s in %v\n", rw.Status(), http.StatusText(rw.Status()), time.Since(start))
}
}

View File

@@ -0,0 +1,31 @@
package martini
import (
"bytes"
"log"
"net/http"
"net/http/httptest"
"testing"
)
func Test_Logger(t *testing.T) {
buff := bytes.NewBufferString("")
recorder := httptest.NewRecorder()
m := New()
// replace log for testing
m.Map(log.New(buff, "[martini] ", 0))
m.Use(Logger())
m.Use(func(res http.ResponseWriter) {
res.WriteHeader(http.StatusNotFound)
})
req, err := http.NewRequest("GET", "http://localhost:3000/foobar", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(recorder, req)
expect(t, recorder.Code, http.StatusNotFound)
refute(t, len(buff.String()), 0)
}

View File

@@ -0,0 +1,173 @@
// Package martini is a powerful package for quickly writing modular web applications/services in Golang.
//
// For a full guide visit http://github.com/codegangsta/martini
//
// package main
//
// import "github.com/codegangsta/martini"
//
// func main() {
// m := martini.Classic()
//
// m.Get("/", func() string {
// return "Hello world!"
// })
//
// m.Run()
// }
package martini
import (
"log"
"net/http"
"os"
"reflect"
"github.com/codegangsta/inject"
)
// Martini represents the top level web application. inject.Injector methods can be invoked to map services on a global level.
type Martini struct {
inject.Injector
handlers []Handler
action Handler
logger *log.Logger
}
// New creates a bare bones Martini instance. Use this method if you want to have full control over the middleware that is used.
func New() *Martini {
m := &Martini{Injector: inject.New(), action: func() {}, logger: log.New(os.Stdout, "[martini] ", 0)}
m.Map(m.logger)
m.Map(defaultReturnHandler())
return m
}
// Handlers sets the entire middleware stack with the given Handlers. This will clear any current middleware handlers.
// Will panic if any of the handlers is not a callable function
func (m *Martini) Handlers(handlers ...Handler) {
m.handlers = make([]Handler, 0)
for _, handler := range handlers {
m.Use(handler)
}
}
// Action sets the handler that will be called after all the middleware has been invoked. This is set to martini.Router in a martini.Classic().
func (m *Martini) Action(handler Handler) {
validateHandler(handler)
m.action = handler
}
// Use adds a middleware Handler to the stack. Will panic if the handler is not a callable func. Middleware Handlers are invoked in the order that they are added.
func (m *Martini) Use(handler Handler) {
validateHandler(handler)
m.handlers = append(m.handlers, handler)
}
// ServeHTTP is the HTTP Entry point for a Martini instance. Useful if you want to control your own HTTP server.
func (m *Martini) ServeHTTP(res http.ResponseWriter, req *http.Request) {
m.createContext(res, req).run()
}
// Run the http server. Listening on os.GetEnv("PORT") or 3000 by default.
func (m *Martini) Run() {
port := os.Getenv("PORT")
if port == "" {
port = "3000"
}
host := os.Getenv("HOST")
m.logger.Println("listening on " + host + ":" + port)
m.logger.Fatalln(http.ListenAndServe(host+":"+port, m))
}
func (m *Martini) createContext(res http.ResponseWriter, req *http.Request) *context {
c := &context{inject.New(), m.handlers, m.action, NewResponseWriter(res), 0}
c.SetParent(m)
c.MapTo(c, (*Context)(nil))
c.MapTo(c.rw, (*http.ResponseWriter)(nil))
c.Map(req)
return c
}
// ClassicMartini represents a Martini with some reasonable defaults. Embeds the router functions for convenience.
type ClassicMartini struct {
*Martini
Router
}
// Classic creates a classic Martini with some basic default middleware - martini.Logger, martini.Recovery and martini.Static.
// Classic also maps martini.Routes as a service.
func Classic() *ClassicMartini {
r := NewRouter()
m := New()
m.Use(Logger())
m.Use(Recovery())
m.Use(Static("public"))
m.MapTo(r, (*Routes)(nil))
m.Action(r.Handle)
return &ClassicMartini{m, r}
}
// Handler can be any callable function. Martini attempts to inject services into the handler's argument list.
// Martini will panic if an argument could not be fullfilled via dependency injection.
type Handler interface{}
func validateHandler(handler Handler) {
if reflect.TypeOf(handler).Kind() != reflect.Func {
panic("martini handler must be a callable func")
}
}
// Context represents a request context. Services can be mapped on the request level from this interface.
type Context interface {
inject.Injector
// Next is an optional function that Middleware Handlers can call to yield the until after
// the other Handlers have been executed. This works really well for any operations that must
// happen after an http request
Next()
// Written returns whether or not the response for this context has been written.
Written() bool
}
type context struct {
inject.Injector
handlers []Handler
action Handler
rw ResponseWriter
index int
}
func (c *context) handler() Handler {
if c.index < len(c.handlers) {
return c.handlers[c.index]
}
if c.index == len(c.handlers) {
return c.action
}
panic("invalid index for context handler")
}
func (c *context) Next() {
c.index += 1
c.run()
}
func (c *context) Written() bool {
return c.rw.Written()
}
func (c *context) run() {
for c.index <= len(c.handlers) {
_, err := c.Invoke(c.handler())
if err != nil {
panic(err)
}
c.index += 1
if c.Written() {
return
}
}
}

View File

@@ -0,0 +1,141 @@
package martini
import (
"net/http"
"net/http/httptest"
"reflect"
"testing"
)
/* Test Helpers */
func expect(t *testing.T, a interface{}, b interface{}) {
if a != b {
t.Errorf("Expected %v (type %v) - Got %v (type %v)", b, reflect.TypeOf(b), a, reflect.TypeOf(a))
}
}
func refute(t *testing.T, a interface{}, b interface{}) {
if a == b {
t.Errorf("Did not expect %v (type %v) - Got %v (type %v)", b, reflect.TypeOf(b), a, reflect.TypeOf(a))
}
}
func Test_New(t *testing.T) {
m := New()
if m == nil {
t.Error("martini.New() cannot return nil")
}
}
func Test_Martini_Run(t *testing.T) {
// just test that Run doesn't bomb
go New().Run()
}
func Test_Martini_ServeHTTP(t *testing.T) {
result := ""
response := httptest.NewRecorder()
m := New()
m.Use(func(c Context) {
result += "foo"
c.Next()
result += "ban"
})
m.Use(func(c Context) {
result += "bar"
c.Next()
result += "baz"
})
m.Action(func(res http.ResponseWriter, req *http.Request) {
result += "bat"
res.WriteHeader(http.StatusBadRequest)
})
m.ServeHTTP(response, (*http.Request)(nil))
expect(t, result, "foobarbatbazban")
expect(t, response.Code, http.StatusBadRequest)
}
func Test_Martini_Handlers(t *testing.T) {
result := ""
response := httptest.NewRecorder()
batman := func(c Context) {
result += "batman!"
}
m := New()
m.Use(func(c Context) {
result += "foo"
c.Next()
result += "ban"
})
m.Handlers(
batman,
batman,
batman,
)
m.Action(func(res http.ResponseWriter, req *http.Request) {
result += "bat"
res.WriteHeader(http.StatusBadRequest)
})
m.ServeHTTP(response, (*http.Request)(nil))
expect(t, result, "batman!batman!batman!bat")
expect(t, response.Code, http.StatusBadRequest)
}
func Test_Martini_EarlyWrite(t *testing.T) {
result := ""
response := httptest.NewRecorder()
m := New()
m.Use(func(res http.ResponseWriter) {
result += "foobar"
res.Write([]byte("Hello world"))
})
m.Use(func() {
result += "bat"
})
m.Action(func(res http.ResponseWriter) {
result += "baz"
res.WriteHeader(http.StatusBadRequest)
})
m.ServeHTTP(response, (*http.Request)(nil))
expect(t, result, "foobar")
expect(t, response.Code, http.StatusOK)
}
func Test_Martini_Written(t *testing.T) {
response := httptest.NewRecorder()
m := New()
m.Handlers(func(res http.ResponseWriter) {
res.WriteHeader(http.StatusOK)
})
ctx := m.createContext(response, (*http.Request)(nil))
expect(t, ctx.Written(), false)
ctx.run()
expect(t, ctx.Written(), true)
}
func Test_Martini_Basic_NoRace(t *testing.T) {
m := New()
handlers := []Handler{func() {}, func() {}}
// Ensure append will not realloc to trigger the race condition
m.handlers = handlers[:1]
req, _ := http.NewRequest("GET", "/", nil)
for i := 0; i < 2; i++ {
go func() {
response := httptest.NewRecorder()
m.ServeHTTP(response, req)
}()
}
}

View File

@@ -0,0 +1,142 @@
package martini
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"net/http"
"runtime"
"github.com/codegangsta/inject"
)
const (
panicHtml = `<html>
<head><title>PANIC: %s</title>
<style type="text/css">
html, body {
font-family: "Roboto", sans-serif;
color: #333333;
background-color: #ea5343;
margin: 0px;
}
h1 {
color: #d04526;
background-color: #ffffff;
padding: 20px;
border-bottom: 1px dashed #2b3848;
}
pre {
margin: 20px;
padding: 20px;
border: 2px solid #2b3848;
background-color: #ffffff;
}
</style>
</head><body>
<h1>PANIC</h1>
<pre style="font-weight: bold;">%s</pre>
<pre>%s</pre>
</body>
</html>`
)
var (
dunno = []byte("???")
centerDot = []byte("·")
dot = []byte(".")
slash = []byte("/")
)
// stack returns a nicely formated stack frame, skipping skip frames
func stack(skip int) []byte {
buf := new(bytes.Buffer) // the returned data
// As we loop, we open files and read them. These variables record the currently
// loaded file.
var lines [][]byte
var lastFile string
for i := skip; ; i++ { // Skip the expected number of frames
pc, file, line, ok := runtime.Caller(i)
if !ok {
break
}
// Print this much at least. If we can't find the source, it won't show.
fmt.Fprintf(buf, "%s:%d (0x%x)\n", file, line, pc)
if file != lastFile {
data, err := ioutil.ReadFile(file)
if err != nil {
continue
}
lines = bytes.Split(data, []byte{'\n'})
lastFile = file
}
fmt.Fprintf(buf, "\t%s: %s\n", function(pc), source(lines, line))
}
return buf.Bytes()
}
// source returns a space-trimmed slice of the n'th line.
func source(lines [][]byte, n int) []byte {
n-- // in stack trace, lines are 1-indexed but our array is 0-indexed
if n < 0 || n >= len(lines) {
return dunno
}
return bytes.TrimSpace(lines[n])
}
// function returns, if possible, the name of the function containing the PC.
func function(pc uintptr) []byte {
fn := runtime.FuncForPC(pc)
if fn == nil {
return dunno
}
name := []byte(fn.Name())
// The name includes the path name to the package, which is unnecessary
// since the file name is already included. Plus, it has center dots.
// That is, we see
// runtime/debug.*T·ptrmethod
// and want
// *T.ptrmethod
// Also the package path might contains dot (e.g. code.google.com/...),
// so first eliminate the path prefix
if lastslash := bytes.LastIndex(name, slash); lastslash >= 0 {
name = name[lastslash+1:]
}
if period := bytes.Index(name, dot); period >= 0 {
name = name[period+1:]
}
name = bytes.Replace(name, centerDot, dot, -1)
return name
}
// Recovery returns a middleware that recovers from any panics and writes a 500 if there was one.
// While Martini is in development mode, Recovery will also output the panic as HTML.
func Recovery() Handler {
return func(c Context, log *log.Logger) {
defer func() {
if err := recover(); err != nil {
stack := stack(3)
log.Printf("PANIC: %s\n%s", err, stack)
// Lookup the current responsewriter
val := c.Get(inject.InterfaceOf((*http.ResponseWriter)(nil)))
res := val.Interface().(http.ResponseWriter)
// respond with panic message while in development mode
var body []byte
if Env == Dev {
res.Header().Set("Content-Type", "text/html")
body = []byte(fmt.Sprintf(panicHtml, err, err, stack))
}
res.WriteHeader(http.StatusInternalServerError)
if nil != body {
res.Write(body)
}
}
}()
c.Next()
}
}

View File

@@ -0,0 +1,49 @@
package martini
import (
"bytes"
"log"
"net/http"
"net/http/httptest"
"testing"
)
func Test_Recovery(t *testing.T) {
buff := bytes.NewBufferString("")
recorder := httptest.NewRecorder()
setENV(Dev)
m := New()
// replace log for testing
m.Map(log.New(buff, "[martini] ", 0))
m.Use(func(res http.ResponseWriter, req *http.Request) {
res.Header().Set("Content-Type", "unpredictable")
})
m.Use(Recovery())
m.Use(func(res http.ResponseWriter, req *http.Request) {
panic("here is a panic!")
})
m.ServeHTTP(recorder, (*http.Request)(nil))
expect(t, recorder.Code, http.StatusInternalServerError)
expect(t, recorder.HeaderMap.Get("Content-Type"), "text/html")
refute(t, recorder.Body.Len(), 0)
refute(t, len(buff.String()), 0)
}
func Test_Recovery_ResponseWriter(t *testing.T) {
recorder := httptest.NewRecorder()
recorder2 := httptest.NewRecorder()
setENV(Dev)
m := New()
m.Use(Recovery())
m.Use(func(c Context) {
c.MapTo(recorder2, (*http.ResponseWriter)(nil))
panic("here is a panic!")
})
m.ServeHTTP(recorder, (*http.Request)(nil))
expect(t, recorder2.Code, http.StatusInternalServerError)
expect(t, recorder2.HeaderMap.Get("Content-Type"), "text/html")
refute(t, recorder2.Body.Len(), 0)
}

View File

@@ -0,0 +1,97 @@
package martini
import (
"bufio"
"fmt"
"net"
"net/http"
)
// ResponseWriter is a wrapper around http.ResponseWriter that provides extra information about
// the response. It is recommended that middleware handlers use this construct to wrap a responsewriter
// if the functionality calls for it.
type ResponseWriter interface {
http.ResponseWriter
http.Flusher
// Status returns the status code of the response or 0 if the response has not been written.
Status() int
// Written returns whether or not the ResponseWriter has been written.
Written() bool
// Size returns the size of the response body.
Size() int
// Before allows for a function to be called before the ResponseWriter has been written to. This is
// useful for setting headers or any other operations that must happen before a response has been written.
Before(BeforeFunc)
}
// BeforeFunc is a function that is called before the ResponseWriter has been written to.
type BeforeFunc func(ResponseWriter)
// NewResponseWriter creates a ResponseWriter that wraps an http.ResponseWriter
func NewResponseWriter(rw http.ResponseWriter) ResponseWriter {
return &responseWriter{rw, 0, 0, nil}
}
type responseWriter struct {
http.ResponseWriter
status int
size int
beforeFuncs []BeforeFunc
}
func (rw *responseWriter) WriteHeader(s int) {
rw.callBefore()
rw.ResponseWriter.WriteHeader(s)
rw.status = s
}
func (rw *responseWriter) Write(b []byte) (int, error) {
if !rw.Written() {
// The status will be StatusOK if WriteHeader has not been called yet
rw.WriteHeader(http.StatusOK)
}
size, err := rw.ResponseWriter.Write(b)
rw.size += size
return size, err
}
func (rw *responseWriter) Status() int {
return rw.status
}
func (rw *responseWriter) Size() int {
return rw.size
}
func (rw *responseWriter) Written() bool {
return rw.status != 0
}
func (rw *responseWriter) Before(before BeforeFunc) {
rw.beforeFuncs = append(rw.beforeFuncs, before)
}
func (rw *responseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
hijacker, ok := rw.ResponseWriter.(http.Hijacker)
if !ok {
return nil, nil, fmt.Errorf("the ResponseWriter doesn't support the Hijacker interface")
}
return hijacker.Hijack()
}
func (rw *responseWriter) CloseNotify() <-chan bool {
return rw.ResponseWriter.(http.CloseNotifier).CloseNotify()
}
func (rw *responseWriter) callBefore() {
for i := len(rw.beforeFuncs) - 1; i >= 0; i-- {
rw.beforeFuncs[i](rw)
}
}
func (rw *responseWriter) Flush() {
flusher, ok := rw.ResponseWriter.(http.Flusher)
if ok {
flusher.Flush()
}
}

View File

@@ -0,0 +1,188 @@
package martini
import (
"bufio"
"io"
"net"
"net/http"
"net/http/httptest"
"testing"
"time"
)
type closeNotifyingRecorder struct {
*httptest.ResponseRecorder
closed chan bool
}
func newCloseNotifyingRecorder() *closeNotifyingRecorder {
return &closeNotifyingRecorder{
httptest.NewRecorder(),
make(chan bool, 1),
}
}
func (c *closeNotifyingRecorder) close() {
c.closed <- true
}
func (c *closeNotifyingRecorder) CloseNotify() <-chan bool {
return c.closed
}
type hijackableResponse struct {
Hijacked bool
}
func newHijackableResponse() *hijackableResponse {
return &hijackableResponse{}
}
func (h *hijackableResponse) Header() http.Header { return nil }
func (h *hijackableResponse) Write(buf []byte) (int, error) { return 0, nil }
func (h *hijackableResponse) WriteHeader(code int) {}
func (h *hijackableResponse) Flush() {}
func (h *hijackableResponse) Hijack() (net.Conn, *bufio.ReadWriter, error) {
h.Hijacked = true
return nil, nil, nil
}
func Test_ResponseWriter_WritingString(t *testing.T) {
rec := httptest.NewRecorder()
rw := NewResponseWriter(rec)
rw.Write([]byte("Hello world"))
expect(t, rec.Code, rw.Status())
expect(t, rec.Body.String(), "Hello world")
expect(t, rw.Status(), http.StatusOK)
expect(t, rw.Size(), 11)
expect(t, rw.Written(), true)
}
func Test_ResponseWriter_WritingStrings(t *testing.T) {
rec := httptest.NewRecorder()
rw := NewResponseWriter(rec)
rw.Write([]byte("Hello world"))
rw.Write([]byte("foo bar bat baz"))
expect(t, rec.Code, rw.Status())
expect(t, rec.Body.String(), "Hello worldfoo bar bat baz")
expect(t, rw.Status(), http.StatusOK)
expect(t, rw.Size(), 26)
}
func Test_ResponseWriter_WritingHeader(t *testing.T) {
rec := httptest.NewRecorder()
rw := NewResponseWriter(rec)
rw.WriteHeader(http.StatusNotFound)
expect(t, rec.Code, rw.Status())
expect(t, rec.Body.String(), "")
expect(t, rw.Status(), http.StatusNotFound)
expect(t, rw.Size(), 0)
}
func Test_ResponseWriter_Before(t *testing.T) {
rec := httptest.NewRecorder()
rw := NewResponseWriter(rec)
result := ""
rw.Before(func(ResponseWriter) {
result += "foo"
})
rw.Before(func(ResponseWriter) {
result += "bar"
})
rw.WriteHeader(http.StatusNotFound)
expect(t, rec.Code, rw.Status())
expect(t, rec.Body.String(), "")
expect(t, rw.Status(), http.StatusNotFound)
expect(t, rw.Size(), 0)
expect(t, result, "barfoo")
}
func Test_ResponseWriter_Hijack(t *testing.T) {
hijackable := newHijackableResponse()
rw := NewResponseWriter(hijackable)
hijacker, ok := rw.(http.Hijacker)
expect(t, ok, true)
_, _, err := hijacker.Hijack()
if err != nil {
t.Error(err)
}
expect(t, hijackable.Hijacked, true)
}
func Test_ResponseWrite_Hijack_NotOK(t *testing.T) {
hijackable := new(http.ResponseWriter)
rw := NewResponseWriter(*hijackable)
hijacker, ok := rw.(http.Hijacker)
expect(t, ok, true)
_, _, err := hijacker.Hijack()
refute(t, err, nil)
}
func Test_ResponseWriter_CloseNotify(t *testing.T) {
rec := newCloseNotifyingRecorder()
rw := NewResponseWriter(rec)
closed := false
notifier := rw.(http.CloseNotifier).CloseNotify()
rec.close()
select {
case <-notifier:
closed = true
case <-time.After(time.Second):
}
expect(t, closed, true)
}
func Test_ResponseWriter_Flusher(t *testing.T) {
rec := httptest.NewRecorder()
rw := NewResponseWriter(rec)
_, ok := rw.(http.Flusher)
expect(t, ok, true)
}
func Test_ResponseWriter_FlusherHandler(t *testing.T) {
// New martini instance
m := Classic()
m.Get("/events", func(w http.ResponseWriter, r *http.Request) {
f, ok := w.(http.Flusher)
expect(t, ok, true)
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
for i := 0; i < 2; i++ {
time.Sleep(10 * time.Millisecond)
io.WriteString(w, "data: Hello\n\n")
f.Flush()
}
})
recorder := httptest.NewRecorder()
r, _ := http.NewRequest("GET", "/events", nil)
m.ServeHTTP(recorder, r)
if recorder.Code != 200 {
t.Error("Response not 200")
}
if recorder.Body.String() != "data: Hello\n\ndata: Hello\n\n" {
t.Error("Didn't receive correct body, got:", recorder.Body.String())
}
}

View File

@@ -0,0 +1,43 @@
package martini
import (
"github.com/codegangsta/inject"
"net/http"
"reflect"
)
// ReturnHandler is a service that Martini provides that is called
// when a route handler returns something. The ReturnHandler is
// responsible for writing to the ResponseWriter based on the values
// that are passed into this function.
type ReturnHandler func(Context, []reflect.Value)
func defaultReturnHandler() ReturnHandler {
return func(ctx Context, vals []reflect.Value) {
rv := ctx.Get(inject.InterfaceOf((*http.ResponseWriter)(nil)))
res := rv.Interface().(http.ResponseWriter)
var responseVal reflect.Value
if len(vals) > 1 && vals[0].Kind() == reflect.Int {
res.WriteHeader(int(vals[0].Int()))
responseVal = vals[1]
} else if len(vals) > 0 {
responseVal = vals[0]
}
if canDeref(responseVal) {
responseVal = responseVal.Elem()
}
if isByteSlice(responseVal) {
res.Write(responseVal.Bytes())
} else {
res.Write([]byte(responseVal.String()))
}
}
}
func isByteSlice(val reflect.Value) bool {
return val.Kind() == reflect.Slice && val.Type().Elem().Kind() == reflect.Uint8
}
func canDeref(val reflect.Value) bool {
return val.Kind() == reflect.Interface || val.Kind() == reflect.Ptr
}

View File

@@ -0,0 +1,331 @@
package martini
import (
"fmt"
"net/http"
"reflect"
"regexp"
"strconv"
)
// Params is a map of name/value pairs for named routes. An instance of martini.Params is available to be injected into any route handler.
type Params map[string]string
// Router is Martini's de-facto routing interface. Supports HTTP verbs, stacked handlers, and dependency injection.
type Router interface {
Routes
// Group adds a group where related routes can be added.
Group(string, func(Router), ...Handler)
// Get adds a route for a HTTP GET request to the specified matching pattern.
Get(string, ...Handler) Route
// Patch adds a route for a HTTP PATCH request to the specified matching pattern.
Patch(string, ...Handler) Route
// Post adds a route for a HTTP POST request to the specified matching pattern.
Post(string, ...Handler) Route
// Put adds a route for a HTTP PUT request to the specified matching pattern.
Put(string, ...Handler) Route
// Delete adds a route for a HTTP DELETE request to the specified matching pattern.
Delete(string, ...Handler) Route
// Options adds a route for a HTTP OPTIONS request to the specified matching pattern.
Options(string, ...Handler) Route
// Head adds a route for a HTTP HEAD request to the specified matching pattern.
Head(string, ...Handler) Route
// Any adds a route for any HTTP method request to the specified matching pattern.
Any(string, ...Handler) Route
// NotFound sets the handlers that are called when a no route matches a request. Throws a basic 404 by default.
NotFound(...Handler)
// Handle is the entry point for routing. This is used as a martini.Handler
Handle(http.ResponseWriter, *http.Request, Context)
}
type router struct {
routes []*route
notFounds []Handler
groups []group
}
type group struct {
pattern string
handlers []Handler
}
// NewRouter creates a new Router instance.
// If you aren't using ClassicMartini, then you can add Routes as a
// service with:
//
// m := martini.New()
// r := martini.NewRouter()
// m.MapTo(r, (*martini.Routes)(nil))
//
// If you are using ClassicMartini, then this is done for you.
func NewRouter() Router {
return &router{notFounds: []Handler{http.NotFound}, groups: make([]group, 0)}
}
func (r *router) Group(pattern string, fn func(Router), h ...Handler) {
r.groups = append(r.groups, group{pattern, h})
fn(r)
r.groups = r.groups[:len(r.groups)-1]
}
func (r *router) Get(pattern string, h ...Handler) Route {
return r.addRoute("GET", pattern, h)
}
func (r *router) Patch(pattern string, h ...Handler) Route {
return r.addRoute("PATCH", pattern, h)
}
func (r *router) Post(pattern string, h ...Handler) Route {
return r.addRoute("POST", pattern, h)
}
func (r *router) Put(pattern string, h ...Handler) Route {
return r.addRoute("PUT", pattern, h)
}
func (r *router) Delete(pattern string, h ...Handler) Route {
return r.addRoute("DELETE", pattern, h)
}
func (r *router) Options(pattern string, h ...Handler) Route {
return r.addRoute("OPTIONS", pattern, h)
}
func (r *router) Head(pattern string, h ...Handler) Route {
return r.addRoute("HEAD", pattern, h)
}
func (r *router) Any(pattern string, h ...Handler) Route {
return r.addRoute("*", pattern, h)
}
func (r *router) Handle(res http.ResponseWriter, req *http.Request, context Context) {
for _, route := range r.routes {
ok, vals := route.Match(req.Method, req.URL.Path)
if ok {
params := Params(vals)
context.Map(params)
route.Handle(context, res)
return
}
}
// no routes exist, 404
c := &routeContext{context, 0, r.notFounds}
context.MapTo(c, (*Context)(nil))
c.run()
}
func (r *router) NotFound(handler ...Handler) {
r.notFounds = handler
}
func (r *router) addRoute(method string, pattern string, handlers []Handler) *route {
if len(r.groups) > 0 {
group := r.groups[len(r.groups)-1]
pattern = group.pattern + pattern
h := make([]Handler, len(group.handlers)+len(handlers))
copy(h, group.handlers)
copy(h[len(group.handlers):], handlers)
handlers = h
}
route := newRoute(method, pattern, handlers)
route.Validate()
r.routes = append(r.routes, route)
return route
}
func (r *router) findRoute(name string) *route {
for _, route := range r.routes {
if route.name == name {
return route
}
}
return nil
}
// Route is an interface representing a Route in Martini's routing layer.
type Route interface {
// URLWith returns a rendering of the Route's url with the given string params.
URLWith([]string) string
Name(string)
}
type route struct {
method string
regex *regexp.Regexp
handlers []Handler
pattern string
name string
}
func newRoute(method string, pattern string, handlers []Handler) *route {
route := route{method, nil, handlers, pattern, ""}
r := regexp.MustCompile(`:[^/#?()\.\\]+`)
pattern = r.ReplaceAllStringFunc(pattern, func(m string) string {
return fmt.Sprintf(`(?P<%s>[^/#?]+)`, m[1:])
})
r2 := regexp.MustCompile(`\*\*`)
var index int
pattern = r2.ReplaceAllStringFunc(pattern, func(m string) string {
index++
return fmt.Sprintf(`(?P<_%d>[^#?]*)`, index)
})
pattern += `\/?`
route.regex = regexp.MustCompile(pattern)
return &route
}
func (r route) MatchMethod(method string) bool {
return r.method == "*" || method == r.method || (method == "HEAD" && r.method == "GET")
}
func (r route) Match(method string, path string) (bool, map[string]string) {
// add Any method matching support
if !r.MatchMethod(method) {
return false, nil
}
matches := r.regex.FindStringSubmatch(path)
if len(matches) > 0 && matches[0] == path {
params := make(map[string]string)
for i, name := range r.regex.SubexpNames() {
if len(name) > 0 {
params[name] = matches[i]
}
}
return true, params
}
return false, nil
}
func (r *route) Validate() {
for _, handler := range r.handlers {
validateHandler(handler)
}
}
func (r *route) Handle(c Context, res http.ResponseWriter) {
context := &routeContext{c, 0, r.handlers}
c.MapTo(context, (*Context)(nil))
context.run()
}
// URLWith returns the url pattern replacing the parameters for its values
func (r *route) URLWith(args []string) string {
if len(args) > 0 {
reg := regexp.MustCompile(`:[^/#?()\.\\]+`)
argCount := len(args)
i := 0
url := reg.ReplaceAllStringFunc(r.pattern, func(m string) string {
var val interface{}
if i < argCount {
val = args[i]
} else {
val = m
}
i += 1
return fmt.Sprintf(`%v`, val)
})
return url
}
return r.pattern
}
func (r *route) Name(name string) {
r.name = name
}
// Routes is a helper service for Martini's routing layer.
type Routes interface {
// URLFor returns a rendered URL for the given route. Optional params can be passed to fulfill named parameters in the route.
URLFor(name string, params ...interface{}) string
// MethodsFor returns an array of methods available for the path
MethodsFor(path string) []string
}
// URLFor returns the url for the given route name.
func (r *router) URLFor(name string, params ...interface{}) string {
route := r.findRoute(name)
if route == nil {
panic("route not found")
}
var args []string
for _, param := range params {
switch v := param.(type) {
case int:
args = append(args, strconv.FormatInt(int64(v), 10))
case string:
args = append(args, v)
default:
if v != nil {
panic("Arguments passed to URLFor must be integers or strings")
}
}
}
return route.URLWith(args)
}
func hasMethod(methods []string, method string) bool {
for _, v := range methods {
if v == method {
return true
}
}
return false
}
// MethodsFor returns all methods available for path
func (r *router) MethodsFor(path string) []string {
methods := []string{}
for _, route := range r.routes {
matches := route.regex.FindStringSubmatch(path)
if len(matches) > 0 && matches[0] == path && !hasMethod(methods, route.method) {
methods = append(methods, route.method)
}
}
return methods
}
type routeContext struct {
Context
index int
handlers []Handler
}
func (r *routeContext) Next() {
r.index += 1
r.run()
}
func (r *routeContext) run() {
for r.index < len(r.handlers) {
handler := r.handlers[r.index]
vals, err := r.Invoke(handler)
if err != nil {
panic(err)
}
r.index += 1
// if the handler returned something, write it to the http response
if len(vals) > 0 {
ev := r.Get(reflect.TypeOf(ReturnHandler(nil)))
handleReturn := ev.Interface().(ReturnHandler)
handleReturn(r, vals)
}
if r.Written() {
return
}
}
}

View File

@@ -0,0 +1,410 @@
package martini
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func Test_Routing(t *testing.T) {
router := NewRouter()
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("GET", "http://localhost:3000/foo", nil)
context := New().createContext(recorder, req)
req2, _ := http.NewRequest("POST", "http://localhost:3000/bar/bat", nil)
context2 := New().createContext(recorder, req2)
req3, _ := http.NewRequest("DELETE", "http://localhost:3000/baz", nil)
context3 := New().createContext(recorder, req3)
req4, _ := http.NewRequest("PATCH", "http://localhost:3000/bar/foo", nil)
context4 := New().createContext(recorder, req4)
req5, _ := http.NewRequest("GET", "http://localhost:3000/fez/this/should/match", nil)
context5 := New().createContext(recorder, req5)
req6, _ := http.NewRequest("PUT", "http://localhost:3000/pop/blah/blah/blah/bap/foo/", nil)
context6 := New().createContext(recorder, req6)
req7, _ := http.NewRequest("DELETE", "http://localhost:3000/wap//pow", nil)
context7 := New().createContext(recorder, req7)
req8, _ := http.NewRequest("HEAD", "http://localhost:3000/wap//pow", nil)
context8 := New().createContext(recorder, req8)
req9, _ := http.NewRequest("OPTIONS", "http://localhost:3000/opts", nil)
context9 := New().createContext(recorder, req9)
req10, _ := http.NewRequest("HEAD", "http://localhost:3000/foo", nil)
context10 := New().createContext(recorder, req10)
req11, _ := http.NewRequest("GET", "http://localhost:3000/bazz/inga", nil)
context11 := New().createContext(recorder, req11)
req12, _ := http.NewRequest("POST", "http://localhost:3000/bazz/inga", nil)
context12 := New().createContext(recorder, req12)
result := ""
router.Get("/foo", func(req *http.Request) {
result += "foo"
})
router.Patch("/bar/:id", func(params Params) {
expect(t, params["id"], "foo")
result += "barfoo"
})
router.Post("/bar/:id", func(params Params) {
expect(t, params["id"], "bat")
result += "barbat"
})
router.Put("/fizzbuzz", func() {
result += "fizzbuzz"
})
router.Delete("/bazzer", func(c Context) {
result += "baz"
})
router.Get("/fez/**", func(params Params) {
expect(t, params["_1"], "this/should/match")
result += "fez"
})
router.Put("/pop/**/bap/:id/**", func(params Params) {
expect(t, params["id"], "foo")
expect(t, params["_1"], "blah/blah/blah")
expect(t, params["_2"], "")
result += "popbap"
})
router.Delete("/wap/**/pow", func(params Params) {
expect(t, params["_1"], "")
result += "wappow"
})
router.Options("/opts", func() {
result += "opts"
})
router.Head("/wap/**/pow", func(params Params) {
expect(t, params["_1"], "")
result += "wappow"
})
router.Group("/bazz", func(r Router) {
r.Get("/inga", func() {
result += "get"
})
r.Post("/inga", func() {
result += "post"
})
}, func() {
result += "bazz"
}, func() {
result += "inga"
})
router.Handle(recorder, req, context)
router.Handle(recorder, req2, context2)
router.Handle(recorder, req3, context3)
router.Handle(recorder, req4, context4)
router.Handle(recorder, req5, context5)
router.Handle(recorder, req6, context6)
router.Handle(recorder, req7, context7)
router.Handle(recorder, req8, context8)
router.Handle(recorder, req9, context9)
router.Handle(recorder, req10, context10)
router.Handle(recorder, req11, context11)
router.Handle(recorder, req12, context12)
expect(t, result, "foobarbatbarfoofezpopbapwappowwappowoptsfoobazzingagetbazzingapost")
expect(t, recorder.Code, http.StatusNotFound)
expect(t, recorder.Body.String(), "404 page not found\n")
}
func Test_RouterHandlerStatusCode(t *testing.T) {
router := NewRouter()
router.Get("/foo", func() string {
return "foo"
})
router.Get("/bar", func() (int, string) {
return http.StatusForbidden, "bar"
})
router.Get("/baz", func() (string, string) {
return "baz", "BAZ!"
})
router.Get("/bytes", func() []byte {
return []byte("Bytes!")
})
router.Get("/interface", func() interface{} {
return "Interface!"
})
// code should be 200 if none is returned from the handler
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("GET", "http://localhost:3000/foo", nil)
context := New().createContext(recorder, req)
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusOK)
expect(t, recorder.Body.String(), "foo")
// if a status code is returned, it should be used
recorder = httptest.NewRecorder()
req, _ = http.NewRequest("GET", "http://localhost:3000/bar", nil)
context = New().createContext(recorder, req)
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusForbidden)
expect(t, recorder.Body.String(), "bar")
// shouldn't use the first returned value as a status code if not an integer
recorder = httptest.NewRecorder()
req, _ = http.NewRequest("GET", "http://localhost:3000/baz", nil)
context = New().createContext(recorder, req)
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusOK)
expect(t, recorder.Body.String(), "baz")
// Should render bytes as a return value as well.
recorder = httptest.NewRecorder()
req, _ = http.NewRequest("GET", "http://localhost:3000/bytes", nil)
context = New().createContext(recorder, req)
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusOK)
expect(t, recorder.Body.String(), "Bytes!")
// Should render interface{} values.
recorder = httptest.NewRecorder()
req, _ = http.NewRequest("GET", "http://localhost:3000/interface", nil)
context = New().createContext(recorder, req)
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusOK)
expect(t, recorder.Body.String(), "Interface!")
}
func Test_RouterHandlerStacking(t *testing.T) {
router := NewRouter()
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("GET", "http://localhost:3000/foo", nil)
context := New().createContext(recorder, req)
result := ""
f1 := func() {
result += "foo"
}
f2 := func(c Context) {
result += "bar"
c.Next()
result += "bing"
}
f3 := func() string {
result += "bat"
return "Hello world"
}
f4 := func() {
result += "baz"
}
router.Get("/foo", f1, f2, f3, f4)
router.Handle(recorder, req, context)
expect(t, result, "foobarbatbing")
expect(t, recorder.Body.String(), "Hello world")
}
var routeTests = []struct {
// in
method string
path string
// out
ok bool
params map[string]string
}{
{"GET", "/foo/123/bat/321", true, map[string]string{"bar": "123", "baz": "321"}},
{"POST", "/foo/123/bat/321", false, map[string]string{}},
{"GET", "/foo/hello/bat/world", true, map[string]string{"bar": "hello", "baz": "world"}},
{"GET", "foo/hello/bat/world", false, map[string]string{}},
{"GET", "/foo/123/bat/321/", true, map[string]string{"bar": "123", "baz": "321"}},
{"GET", "/foo/123/bat/321//", false, map[string]string{}},
{"GET", "/foo/123//bat/321/", false, map[string]string{}},
}
func Test_RouteMatching(t *testing.T) {
route := newRoute("GET", "/foo/:bar/bat/:baz", nil)
for _, tt := range routeTests {
ok, params := route.Match(tt.method, tt.path)
if ok != tt.ok || params["bar"] != tt.params["bar"] || params["baz"] != tt.params["baz"] {
t.Errorf("expected: (%v, %v) got: (%v, %v)", tt.ok, tt.params, ok, params)
}
}
}
func Test_MethodsFor(t *testing.T) {
router := NewRouter()
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("POST", "http://localhost:3000/foo", nil)
context := New().createContext(recorder, req)
context.MapTo(router, (*Routes)(nil))
router.Post("/foo/bar", func() {
})
router.Post("/fo", func() {
})
router.Get("/foo", func() {
})
router.Put("/foo", func() {
})
router.NotFound(func(routes Routes, w http.ResponseWriter, r *http.Request) {
methods := routes.MethodsFor(r.URL.Path)
if len(methods) != 0 {
w.Header().Set("Allow", strings.Join(methods, ","))
w.WriteHeader(http.StatusMethodNotAllowed)
}
})
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusMethodNotAllowed)
expect(t, recorder.Header().Get("Allow"), "GET,PUT")
}
func Test_NotFound(t *testing.T) {
router := NewRouter()
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("GET", "http://localhost:3000/foo", nil)
context := New().createContext(recorder, req)
router.NotFound(func(res http.ResponseWriter) {
http.Error(res, "Nope", http.StatusNotFound)
})
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusNotFound)
expect(t, recorder.Body.String(), "Nope\n")
}
func Test_NotFoundAsHandler(t *testing.T) {
router := NewRouter()
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("GET", "http://localhost:3000/foo", nil)
context := New().createContext(recorder, req)
router.NotFound(func() string {
return "not found"
})
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusOK)
expect(t, recorder.Body.String(), "not found")
recorder = httptest.NewRecorder()
context = New().createContext(recorder, req)
router.NotFound(func() (int, string) {
return 404, "not found"
})
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusNotFound)
expect(t, recorder.Body.String(), "not found")
recorder = httptest.NewRecorder()
context = New().createContext(recorder, req)
router.NotFound(func() (int, string) {
return 200, ""
})
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusOK)
expect(t, recorder.Body.String(), "")
}
func Test_NotFoundStacking(t *testing.T) {
router := NewRouter()
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("GET", "http://localhost:3000/foo", nil)
context := New().createContext(recorder, req)
result := ""
f1 := func() {
result += "foo"
}
f2 := func(c Context) {
result += "bar"
c.Next()
result += "bing"
}
f3 := func() string {
result += "bat"
return "Not Found"
}
f4 := func() {
result += "baz"
}
router.NotFound(f1, f2, f3, f4)
router.Handle(recorder, req, context)
expect(t, result, "foobarbatbing")
expect(t, recorder.Body.String(), "Not Found")
}
func Test_Any(t *testing.T) {
router := NewRouter()
router.Any("/foo", func(res http.ResponseWriter) {
http.Error(res, "Nope", http.StatusNotFound)
})
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("GET", "http://localhost:3000/foo", nil)
context := New().createContext(recorder, req)
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusNotFound)
expect(t, recorder.Body.String(), "Nope\n")
recorder = httptest.NewRecorder()
req, _ = http.NewRequest("PUT", "http://localhost:3000/foo", nil)
context = New().createContext(recorder, req)
router.Handle(recorder, req, context)
expect(t, recorder.Code, http.StatusNotFound)
expect(t, recorder.Body.String(), "Nope\n")
}
func Test_URLFor(t *testing.T) {
router := NewRouter()
router.Get("/foo", func() {
// Nothing
}).Name("foo")
router.Post("/bar/:id", func(params Params) {
// Nothing
}).Name("bar")
router.Get("/bar/:id/:name", func(params Params, routes Routes) {
expect(t, routes.URLFor("foo", nil), "/foo")
expect(t, routes.URLFor("bar", 5), "/bar/5")
expect(t, routes.URLFor("bar_id", 5, "john"), "/bar/5/john")
}).Name("bar_id")
// code should be 200 if none is returned from the handler
recorder := httptest.NewRecorder()
req, _ := http.NewRequest("GET", "http://localhost:3000/bar/foo/bar", nil)
context := New().createContext(recorder, req)
context.MapTo(router, (*Routes)(nil))
router.Handle(recorder, req, context)
}

View File

@@ -0,0 +1,109 @@
package martini
import (
"log"
"net/http"
"path"
"strings"
)
// StaticOptions is a struct for specifying configuration options for the martini.Static middleware.
type StaticOptions struct {
// Prefix is the optional prefix used to serve the static directory content
Prefix string
// SkipLogging will disable [Static] log messages when a static file is served.
SkipLogging bool
// IndexFile defines which file to serve as index if it exists.
IndexFile string
// Expires defines which user-defined function to use for producing a HTTP Expires Header
// https://developers.google.com/speed/docs/insights/LeverageBrowserCaching
Expires func() string
}
func prepareStaticOptions(options []StaticOptions) StaticOptions {
var opt StaticOptions
if len(options) > 0 {
opt = options[0]
}
// Defaults
if len(opt.IndexFile) == 0 {
opt.IndexFile = "index.html"
}
// Normalize the prefix if provided
if opt.Prefix != "" {
// Ensure we have a leading '/'
if opt.Prefix[0] != '/' {
opt.Prefix = "/" + opt.Prefix
}
// Remove any trailing '/'
opt.Prefix = strings.TrimRight(opt.Prefix, "/")
}
return opt
}
// Static returns a middleware handler that serves static files in the given directory.
func Static(directory string, staticOpt ...StaticOptions) Handler {
dir := http.Dir(directory)
opt := prepareStaticOptions(staticOpt)
return func(res http.ResponseWriter, req *http.Request, log *log.Logger) {
if req.Method != "GET" && req.Method != "HEAD" {
return
}
file := req.URL.Path
// if we have a prefix, filter requests by stripping the prefix
if opt.Prefix != "" {
if !strings.HasPrefix(file, opt.Prefix) {
return
}
file = file[len(opt.Prefix):]
if file != "" && file[0] != '/' {
return
}
}
f, err := dir.Open(file)
if err != nil {
// discard the error?
return
}
defer f.Close()
fi, err := f.Stat()
if err != nil {
return
}
// try to serve index file
if fi.IsDir() {
// redirect if missing trailing slash
if !strings.HasSuffix(req.URL.Path, "/") {
http.Redirect(res, req, req.URL.Path+"/", http.StatusFound)
return
}
file = path.Join(file, opt.IndexFile)
f, err = dir.Open(file)
if err != nil {
return
}
defer f.Close()
fi, err = f.Stat()
if err != nil || fi.IsDir() {
return
}
}
if !opt.SkipLogging {
log.Println("[Static] Serving " + file)
}
// Add an Expires header to the static content
if opt.Expires != nil {
res.Header().Set("Expires", opt.Expires())
}
http.ServeContent(res, req, file, fi.ModTime(), f)
}
}

View File

@@ -0,0 +1,200 @@
package martini
import (
"bytes"
"log"
"net/http"
"net/http/httptest"
"testing"
"github.com/codegangsta/inject"
)
func Test_Static(t *testing.T) {
response := httptest.NewRecorder()
response.Body = new(bytes.Buffer)
m := New()
r := NewRouter()
m.Use(Static("."))
m.Action(r.Handle)
req, err := http.NewRequest("GET", "http://localhost:3000/martini.go", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
expect(t, response.Code, http.StatusOK)
expect(t, response.Header().Get("Expires"), "")
if response.Body.Len() == 0 {
t.Errorf("Got empty body for GET request")
}
}
func Test_Static_Head(t *testing.T) {
response := httptest.NewRecorder()
response.Body = new(bytes.Buffer)
m := New()
r := NewRouter()
m.Use(Static("."))
m.Action(r.Handle)
req, err := http.NewRequest("HEAD", "http://localhost:3000/martini.go", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
expect(t, response.Code, http.StatusOK)
if response.Body.Len() != 0 {
t.Errorf("Got non-empty body for HEAD request")
}
}
func Test_Static_As_Post(t *testing.T) {
response := httptest.NewRecorder()
m := New()
r := NewRouter()
m.Use(Static("."))
m.Action(r.Handle)
req, err := http.NewRequest("POST", "http://localhost:3000/martini.go", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
expect(t, response.Code, http.StatusNotFound)
}
func Test_Static_BadDir(t *testing.T) {
response := httptest.NewRecorder()
m := Classic()
req, err := http.NewRequest("GET", "http://localhost:3000/martini.go", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
refute(t, response.Code, http.StatusOK)
}
func Test_Static_Options_Logging(t *testing.T) {
response := httptest.NewRecorder()
var buffer bytes.Buffer
m := &Martini{Injector: inject.New(), action: func() {}, logger: log.New(&buffer, "[martini] ", 0)}
m.Map(m.logger)
m.Map(defaultReturnHandler())
opt := StaticOptions{}
m.Use(Static(".", opt))
req, err := http.NewRequest("GET", "http://localhost:3000/martini.go", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
expect(t, response.Code, http.StatusOK)
expect(t, buffer.String(), "[martini] [Static] Serving /martini.go\n")
// Now without logging
m.Handlers()
buffer.Reset()
// This should disable logging
opt.SkipLogging = true
m.Use(Static(".", opt))
m.ServeHTTP(response, req)
expect(t, response.Code, http.StatusOK)
expect(t, buffer.String(), "")
}
func Test_Static_Options_ServeIndex(t *testing.T) {
response := httptest.NewRecorder()
var buffer bytes.Buffer
m := &Martini{Injector: inject.New(), action: func() {}, logger: log.New(&buffer, "[martini] ", 0)}
m.Map(m.logger)
m.Map(defaultReturnHandler())
opt := StaticOptions{IndexFile: "martini.go"} // Define martini.go as index file
m.Use(Static(".", opt))
req, err := http.NewRequest("GET", "http://localhost:3000/", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
expect(t, response.Code, http.StatusOK)
expect(t, buffer.String(), "[martini] [Static] Serving /martini.go\n")
}
func Test_Static_Options_Prefix(t *testing.T) {
response := httptest.NewRecorder()
var buffer bytes.Buffer
m := &Martini{Injector: inject.New(), action: func() {}, logger: log.New(&buffer, "[martini] ", 0)}
m.Map(m.logger)
m.Map(defaultReturnHandler())
// Serve current directory under /public
m.Use(Static(".", StaticOptions{Prefix: "/public"}))
// Check file content behaviour
req, err := http.NewRequest("GET", "http://localhost:3000/public/martini.go", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
expect(t, response.Code, http.StatusOK)
expect(t, buffer.String(), "[martini] [Static] Serving /martini.go\n")
}
func Test_Static_Options_Expires(t *testing.T) {
response := httptest.NewRecorder()
var buffer bytes.Buffer
m := &Martini{Injector: inject.New(), action: func() {}, logger: log.New(&buffer, "[martini] ", 0)}
m.Map(m.logger)
m.Map(defaultReturnHandler())
// Serve current directory under /public
m.Use(Static(".", StaticOptions{Expires: func() string { return "46" }}))
// Check file content behaviour
req, err := http.NewRequest("GET", "http://localhost:3000/martini.go", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
expect(t, response.Header().Get("Expires"), "46")
}
func Test_Static_Redirect(t *testing.T) {
response := httptest.NewRecorder()
m := New()
m.Use(Static(".", StaticOptions{Prefix: "/public"}))
req, err := http.NewRequest("GET", "http://localhost:3000/public", nil)
if err != nil {
t.Error(err)
}
m.ServeHTTP(response, req)
expect(t, response.Code, http.StatusFound)
expect(t, response.Header().Get("Location"), "/public/")
}

View File

@@ -0,0 +1,311 @@
# Martini [![wercker status](https://app.wercker.com/status/174bef7e3c999e103cacfe2770102266 "wercker status")](https://app.wercker.com/project/bykey/174bef7e3c999e103cacfe2770102266) [![GoDoc](https://godoc.org/github.com/codegangsta/martini?status.png)](http://godoc.org/github.com/codegangsta/martini)
Martini是一个强大为了编写模块化Web应用而生的GO语言框架.
## 第一个应用
在你安装了GO语言和设置了你的[GOPATH](http://golang.org/doc/code.html#GOPATH)之后, 创建你的自己的`.go`文件, 这里我们假设它的名字叫做 `server.go`.
~~~ go
package main
import "github.com/codegangsta/martini"
func main() {
m := martini.Classic()
m.Get("/", func() string {
return "Hello world!"
})
m.Run()
}
~~~
然后安装Martini的包. (注意Martini需要Go语言1.1或者以上的版本支持):
~~~
go get github.com/codegangsta/martini
~~~
最后运行你的服务:
~~~
go run server.go
~~~
这时你将会有一个Martini的服务监听了, 地址是: `localhost:3000`.
## 获得帮助
请加入: [邮件列表](https://groups.google.com/forum/#!forum/martini-go)
或者可以查看在线演示地址: [演示视频](http://martini.codegangsta.io/#demo)
## 功能列表
* 使用极其简单.
* 无侵入式的设计.
* 很好的与其他的Go语言包协同使用.
* 超赞的路径匹配和路由.
* 模块化的设计 - 容易插入功能件,也容易将其拔出来.
* 已有很多的中间件可以直接使用.
* 框架内已拥有很好的开箱即用的功能支持.
* **完全兼容[http.HandlerFunc](http://godoc.org/net/http#HandlerFunc)接口.**
## 更多中间件
更多的中间件和功能组件, 请查看代码仓库: [martini-contrib](https://github.com/martini-contrib).
## 目录
* [核心 Martini](#classic-martini)
* [处理器](#handlers)
* [路由](#routing)
* [服务](#services)
* [服务静态文件](#serving-static-files)
* [中间件处理器](#middleware-handlers)
* [Next()](#next)
* [常见问答](#faq)
## 核心 Martini
为了更快速的启用Martini, [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic) 提供了一些默认的方便Web开发的工具:
~~~ go
m := martini.Classic()
// ... middleware and routing goes here
m.Run()
~~~
下面是Martini核心已经包含的功能 [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic):
* Request/Response Logging (请求/相应日志) - [martini.Logger](http://godoc.org/github.com/codegangsta/martini#Logger)
* Panic Recovery (容错) - [martini.Recovery](http://godoc.org/github.com/codegangsta/martini#Recovery)
* Static File serving (静态文件服务) - [martini.Static](http://godoc.org/github.com/codegangsta/martini#Static)
* Routing (路由) - [martini.Router](http://godoc.org/github.com/codegangsta/martini#Router)
### 处理器
处理器是Martini的灵魂和核心所在. 一个处理器基本上可以是任何的函数:
~~~ go
m.Get("/", func() {
println("hello world")
})
~~~
#### 返回值
当一个处理器返回结果的时候, Martini将会把返回值作为字符串写入到当前的[http.ResponseWriter](http://godoc.org/net/http#ResponseWriter)里面:
~~~ go
m.Get("/", func() string {
return "hello world" // HTTP 200 : "hello world"
})
~~~
另外你也可以选择性的返回多一个状态码:
~~~ go
m.Get("/", func() (int, string) {
return 418, "i'm a teapot" // HTTP 418 : "i'm a teapot"
})
~~~
#### 服务的注入
处理器是通过反射来调用的. Martini 通过*Dependency Injection* *(依赖注入)* 来为处理器注入参数列表. **这样使得Martini与Go语言的`http.HandlerFunc`接口完全兼容.**
如果你加入一个参数到你的处理器, Martini将会搜索它参数列表中的服务并且通过类型判断来解决依赖关系:
~~~ go
m.Get("/", func(res http.ResponseWriter, req *http.Request) { // res 和 req 是通过Martini注入的
res.WriteHeader(200) // HTTP 200
})
~~~
下面的这些服务已经被包含在核心Martini中: [martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic):
* [*log.Logger](http://godoc.org/log#Logger) - Martini的全局日志.
* [martini.Context](http://godoc.org/github.com/codegangsta/martini#Context) - http request context (请求上下文).
* [martini.Params](http://godoc.org/github.com/codegangsta/martini#Params) - `map[string]string` of named params found by route matching. (名字和参数键值对的参数列表)
* [martini.Routes](http://godoc.org/github.com/codegangsta/martini#Routes) - Route helper service. (路由协助处理)
* [http.ResponseWriter](http://godoc.org/net/http/#ResponseWriter) - http Response writer interface. (响应结果的流接口)
* [*http.Request](http://godoc.org/net/http/#Request) - http Request. http请求)
### 路由
在Martini中, 路由是一个HTTP方法配对一个URL匹配模型. 每一个路由可以对应一个或多个处理器方法:
~~~ go
m.Get("/", func() {
// 显示
})
m.Patch("/", func() {
// 更新
})
m.Post("/", func() {
// 创建
})
m.Put("/", func() {
// 替换
})
m.Delete("/", func() {
// 删除
})
m.Options("/", func() {
// http 选项
})
m.NotFound(func() {
// 处理 404
})
~~~
路由匹配的顺序是按照他们被定义的顺序执行的. 最先被定义的路由将会首先被用户请求匹配并调用.
路由模型可能包含参数列表, 可以通过[martini.Params](http://godoc.org/github.com/codegangsta/martini#Params)服务来获取:
~~~ go
m.Get("/hello/:name", func(params martini.Params) string {
return "Hello " + params["name"]
})
~~~
路由匹配可以通过正则表达式或者glob的形式:
~~~ go
m.Get("/hello/**", func(params martini.Params) string {
return "Hello " + params["_1"]
})
~~~
路由处理器可以被相互叠加使用, 例如很有用的地方可以是在验证和授权的时候:
~~~ go
m.Get("/secret", authorize, func() {
// 该方法将会在authorize方法没有输出结果的时候执行.
})
~~~
### 服务
服务即是被注入到处理器中的参数. 你可以映射一个服务到 *全局* 或者 *请求* 的级别.
#### 全局映射
如果一个Martini实现了inject.Injector的接口, 那么映射成为一个服务就非常简单:
~~~ go
db := &MyDatabase{}
m := martini.Classic()
m.Map(db) // *MyDatabase 这个服务将可以在所有的处理器中被使用到.
// ...
m.Run()
~~~
#### 请求级别的映射
映射在请求级别的服务可以用[martini.Context](http://godoc.org/github.com/codegangsta/martini#Context)来完成:
~~~ go
func MyCustomLoggerHandler(c martini.Context, req *http.Request) {
logger := &MyCustomLogger{req}
c.Map(logger) // 映射成为了 *MyCustomLogger
}
~~~
#### 映射值到接口
关于服务最强悍的地方之一就是它能够映射服务到接口. 例如说, 假设你想要覆盖[http.ResponseWriter](http://godoc.org/net/http#ResponseWriter)成为一个对象, 那么你可以封装它并包含你自己的额外操作, 你可以如下这样来编写你的处理器:
~~~ go
func WrapResponseWriter(res http.ResponseWriter, c martini.Context) {
rw := NewSpecialResponseWriter(res)
c.MapTo(rw, (*http.ResponseWriter)(nil)) // 覆盖 ResponseWriter 成为我们封装过的 ResponseWriter
}
~~~
### 服务静态文件
[martini.Classic()](http://godoc.org/github.com/codegangsta/martini#Classic) 默认会服务位于你服务器环境根目录下的"public"文件夹.
你可以通过加入[martini.Static](http://godoc.org/github.com/codegangsta/martini#Static)的处理器来加入更多的静态文件服务的文件夹.
~~~ go
m.Use(martini.Static("assets")) // 也会服务静态文件于"assets"的文件夹
~~~
## 中间件处理器
中间件处理器是工作于请求和路由之间的. 本质上来说和Martini其他的处理器没有分别. 你可以像如下这样添加一个中间件处理器到它的堆中:
~~~ go
m.Use(func() {
// 做一些中间件该做的事情
})
~~~
你可以通过`Handlers`函数对中间件堆有完全的控制. 它将会替换掉之前的任何设置过的处理器:
~~~ go
m.Handlers(
Middleware1,
Middleware2,
Middleware3,
)
~~~
中间件处理器可以非常好处理一些功能像logging(日志), authorization(授权), authentication(认证), sessions(会话), error pages(错误页面), 以及任何其他的操作需要在http请求发生之前或者之后的:
~~~ go
// 验证api密匙
m.Use(func(res http.ResponseWriter, req *http.Request) {
if req.Header.Get("X-API-KEY") != "secret123" {
res.WriteHeader(http.StatusUnauthorized)
}
})
~~~
### Next()
[Context.Next()](http://godoc.org/github.com/codegangsta/martini#Context)是一个可选的函数用于中间件处理器暂时放弃执行直到其他的处理器都执行完毕. 这样就可以很好的处理在http请求完成后需要做的操作.
~~~ go
// log 记录请求完成前后 (*译者注: 很巧妙,掌声鼓励.)
m.Use(func(c martini.Context, log *log.Logger){
log.Println("before a request")
c.Next()
log.Println("after a request")
})
~~~
## 常见问答
### 我在哪里可以找到中间件资源?
可以查看 [martini-contrib](https://github.com/martini-contrib) 项目. 如果看了觉得没有什么好货色, 可以联系martini-contrib的团队成员为你创建一个新的代码资源库.
* [auth](https://github.com/martini-contrib/auth) - 认证处理器.
* [binding](https://github.com/martini-contrib/binding) - 映射/验证raw请求到结构体(structure)里的处理器
* [gzip](https://github.com/martini-contrib/gzip) - 加入giz支持的处理器
* [render](https://github.com/martini-contrib/render) - 渲染JSON和HTML模板的处理器.
* [acceptlang](https://github.com/martini-contrib/acceptlang) - 解析`Accept-Language` HTTP报头的处理器.
* [sessions](https://github.com/martini-contrib/sessions) - 提供会话服务支持的处理器.
* [strip](https://github.com/martini-contrib/strip) - URL Prefix stripping.
* [method](https://github.com/martini-contrib/method) - HTTP method overriding via Header or form fields.
* [secure](https://github.com/martini-contrib/secure) - Implements a few quick security wins.
* [encoder](https://github.com/martini-contrib/encoder) - Encoder service for rendering data in several formats and content negotiation.
### 我如何整合到我现有的服务器中?
由于Martini实现了 `http.Handler`, 所以它可以很简单的应用到现有Go服务器的子集中. 例如说这是一段在Google App Engine中的示例:
~~~ go
package hello
import (
"net/http"
"github.com/codegangsta/martini"
)
func init() {
m := martini.Classic()
m.Get("/", func() string {
return "Hello world!"
})
http.Handle("/", m)
}
~~~
### 我如何修改port/host?
Martini的`Run`函数会检查PORT和HOST的环境变量并使用它们. 否则Martini将会默认使用localhost:3000
如果想要自定义PORT和HOST, 使用`http.ListenAndServe`函数来代替.
~~~ go
m := martini.Classic()
// ...
http.ListenAndServe(":8080", m)
~~~
## 贡献
Martini项目想要保持简单且干净的代码. 大部分的代码应该贡献到[martini-contrib](https://github.com/martini-contrib)组织中作为一个项目. 如果你想要贡献Martini的核心代码也可以发起一个Pull Request.
## 关于
灵感来自于 [express](https://github.com/visionmedia/express) 和 [sinatra](https://github.com/sinatra/sinatra)
Martini作者 [Code Gangsta](http://codegangsta.io/)
译者: [Leon](http://github.com/leonli)

View File

@@ -0,0 +1 @@
box: wercker/golang@1.1.1

View File

@@ -0,0 +1,121 @@
/*
Copyright 2013 Google Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
// Package lru implements an LRU cache.
package lru
import "container/list"
// Cache is an LRU cache. It is not safe for concurrent access.
type Cache struct {
// MaxEntries is the maximum number of cache entries before
// an item is evicted. Zero means no limit.
MaxEntries int
// OnEvicted optionally specificies a callback function to be
// executed when an entry is purged from the cache.
OnEvicted func(key Key, value interface{})
ll *list.List
cache map[interface{}]*list.Element
}
// A Key may be any value that is comparable. See http://golang.org/ref/spec#Comparison_operators
type Key interface{}
type entry struct {
key Key
value interface{}
}
// New creates a new Cache.
// If maxEntries is zero, the cache has no limit and it's assumed
// that eviction is done by the caller.
func New(maxEntries int) *Cache {
return &Cache{
MaxEntries: maxEntries,
ll: list.New(),
cache: make(map[interface{}]*list.Element),
}
}
// Add adds a value to the cache.
func (c *Cache) Add(key Key, value interface{}) {
if c.cache == nil {
c.cache = make(map[interface{}]*list.Element)
c.ll = list.New()
}
if ee, ok := c.cache[key]; ok {
c.ll.MoveToFront(ee)
ee.Value.(*entry).value = value
return
}
ele := c.ll.PushFront(&entry{key, value})
c.cache[key] = ele
if c.MaxEntries != 0 && c.ll.Len() > c.MaxEntries {
c.RemoveOldest()
}
}
// Get looks up a key's value from the cache.
func (c *Cache) Get(key Key) (value interface{}, ok bool) {
if c.cache == nil {
return
}
if ele, hit := c.cache[key]; hit {
c.ll.MoveToFront(ele)
return ele.Value.(*entry).value, true
}
return
}
// Remove removes the provided key from the cache.
func (c *Cache) Remove(key Key) {
if c.cache == nil {
return
}
if ele, hit := c.cache[key]; hit {
c.removeElement(ele)
}
}
// RemoveOldest removes the oldest item from the cache.
func (c *Cache) RemoveOldest() {
if c.cache == nil {
return
}
ele := c.ll.Back()
if ele != nil {
c.removeElement(ele)
}
}
func (c *Cache) removeElement(e *list.Element) {
c.ll.Remove(e)
kv := e.Value.(*entry)
delete(c.cache, kv.key)
if c.OnEvicted != nil {
c.OnEvicted(kv.key, kv.value)
}
}
// Len returns the number of items in the cache.
func (c *Cache) Len() int {
if c.cache == nil {
return 0
}
return c.ll.Len()
}

View File

@@ -0,0 +1,73 @@
/*
Copyright 2013 Google Inc.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package lru
import (
"testing"
)
type simpleStruct struct {
int
string
}
type complexStruct struct {
int
simpleStruct
}
var getTests = []struct {
name string
keyToAdd interface{}
keyToGet interface{}
expectedOk bool
}{
{"string_hit", "myKey", "myKey", true},
{"string_miss", "myKey", "nonsense", false},
{"simple_struct_hit", simpleStruct{1, "two"}, simpleStruct{1, "two"}, true},
{"simeple_struct_miss", simpleStruct{1, "two"}, simpleStruct{0, "noway"}, false},
{"complex_struct_hit", complexStruct{1, simpleStruct{2, "three"}},
complexStruct{1, simpleStruct{2, "three"}}, true},
}
func TestGet(t *testing.T) {
for _, tt := range getTests {
lru := New(0)
lru.Add(tt.keyToAdd, 1234)
val, ok := lru.Get(tt.keyToGet)
if ok != tt.expectedOk {
t.Fatalf("%s: cache hit = %v; want %v", tt.name, ok, !ok)
} else if ok && val != 1234 {
t.Fatalf("%s expected get to return 1234 but got %v", tt.name, val)
}
}
}
func TestRemove(t *testing.T) {
lru := New(0)
lru.Add("myKey", 1234)
if val, ok := lru.Get("myKey"); !ok {
t.Fatal("TestRemove returned no match")
} else if val != 1234 {
t.Fatalf("TestRemove failed. Expected %d, got %v", 1234, val)
}
lru.Remove("myKey")
if _, ok := lru.Get("myKey"); ok {
t.Fatal("TestRemove returned a removed entry")
}
}

185
Godeps/_workspace/src/github.com/juju/ratelimit/LICENSE generated vendored Normal file
View File

@@ -0,0 +1,185 @@
This software is licensed under the LGPLv3, included below.
As a special exception to the GNU Lesser General Public License version 3
("LGPL3"), the copyright holders of this Library give you permission to
convey to a third party a Combined Work that links statically or dynamically
to this Library without providing any Minimal Corresponding Source or
Minimal Application Code as set out in 4d or providing the installation
information set out in section 4e, provided that you comply with the other
provisions of LGPL3 and provided that you meet, for the Application the
terms and conditions of the license(s) which apply to the Application.
Except as stated in this special exception, the provisions of LGPL3 will
continue to comply in full to this Library. If you modify this Library, you
may apply this exception to your version of this Library, but you are not
obliged to do so. If you do not wish to do so, delete this exception
statement from your version. This exception does not (and cannot) modify any
license terms which apply to the Application, with which you must still
comply.
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

View File

@@ -0,0 +1,109 @@
# ratelimit
--
import "github.com/juju/ratelimit"
The ratelimit package provides an efficient token bucket implementation. See
http://en.wikipedia.org/wiki/Token_bucket.
## Usage
#### func Reader
```go
func Reader(r io.Reader, bucket *Bucket) io.Reader
```
Reader returns a reader that is rate limited by the given token bucket. Each
token in the bucket represents one byte.
#### func Writer
```go
func Writer(w io.Writer, bucket *Bucket) io.Writer
```
Writer returns a reader that is rate limited by the given token bucket. Each
token in the bucket represents one byte.
#### type Bucket
```go
type Bucket struct {
}
```
Bucket represents a token bucket that fills at a predetermined rate. Methods on
Bucket may be called concurrently.
#### func NewBucket
```go
func NewBucket(fillInterval time.Duration, capacity int64) *Bucket
```
NewBucket returns a new token bucket that fills at the rate of one token every
fillInterval, up to the given maximum capacity. Both arguments must be positive.
The bucket is initially full.
#### func NewBucketWithRate
```go
func NewBucketWithRate(rate float64, capacity int64) *Bucket
```
NewBucketWithRate returns a token bucket that fills the bucket at the rate of
rate tokens per second up to the given maximum capacity. Because of limited
clock resolution, at high rates, the actual rate may be up to 1% different from
the specified rate.
#### func (*Bucket) Rate
```go
func (tb *Bucket) Rate() float64
```
Rate returns the fill rate of the bucket, in tokens per second.
#### func (*Bucket) Take
```go
func (tb *Bucket) Take(count int64) time.Duration
```
Take takes count tokens from the bucket without blocking. It returns the time
that the caller should wait until the tokens are actually available.
Note that if the request is irrevocable - there is no way to return tokens to
the bucket once this method commits us to taking them.
#### func (*Bucket) TakeAvailable
```go
func (tb *Bucket) TakeAvailable(count int64) int64
```
TakeAvailable takes up to count immediately available tokens from the bucket. It
returns the number of tokens removed, or zero if there are no available tokens.
It does not block.
#### func (*Bucket) TakeMaxDuration
```go
func (tb *Bucket) TakeMaxDuration(count int64, maxWait time.Duration) (time.Duration, bool)
```
TakeMaxDuration is like Take, except that it will only take tokens from the
bucket if the wait time for the tokens is no greater than maxWait.
If it would take longer than maxWait for the tokens to become available, it does
nothing and reports false, otherwise it returns the time that the caller should
wait until the tokens are actually available, and reports true.
#### func (*Bucket) Wait
```go
func (tb *Bucket) Wait(count int64)
```
Wait takes count tokens from the bucket, waiting until they are available.
#### func (*Bucket) WaitMaxDuration
```go
func (tb *Bucket) WaitMaxDuration(count int64, maxWait time.Duration) bool
```
WaitMaxDuration is like Wait except that it will only take tokens from the
bucket if it needs to wait for no greater than maxWait. It reports whether any
tokens have been removed from the bucket If no tokens have been removed, it
returns immediately.

View File

@@ -0,0 +1,227 @@
// Copyright 2014 Canonical Ltd.
// Licensed under the LGPLv3 with static-linking exception.
// See LICENCE file for details.
// The ratelimit package provides an efficient token bucket implementation.
// See http://en.wikipedia.org/wiki/Token_bucket.
package ratelimit
import (
"strconv"
"sync"
"time"
)
// Bucket represents a token bucket that fills at a predetermined rate.
// Methods on Bucket may be called concurrently.
type Bucket struct {
startTime time.Time
capacity int64
quantum int64
fillInterval time.Duration
// The mutex guards the fields following it.
mu sync.Mutex
// avail holds the number of available tokens
// in the bucket, as of availTick ticks from startTime.
// It will be negative when there are consumers
// waiting for tokens.
avail int64
availTick int64
}
// NewBucket returns a new token bucket that fills at the
// rate of one token every fillInterval, up to the given
// maximum capacity. Both arguments must be
// positive. The bucket is initially full.
func NewBucket(fillInterval time.Duration, capacity int64) *Bucket {
return newBucketWithQuantum(fillInterval, capacity, 1)
}
// rateMargin specifes the allowed variance of actual
// rate from specified rate. 1% seems reasonable.
const rateMargin = 0.01
// NewBucketWithRate returns a token bucket that fills the bucket
// at the rate of rate tokens per second up to the given
// maximum capacity. Because of limited clock resolution,
// at high rates, the actual rate may be up to 1% different from the
// specified rate.
func NewBucketWithRate(rate float64, capacity int64) *Bucket {
for quantum := int64(1); quantum < 1<<50; quantum = nextQuantum(quantum) {
fillInterval := time.Duration(1e9 * float64(quantum) / rate)
if fillInterval <= 0 {
continue
}
tb := newBucketWithQuantum(fillInterval, capacity, quantum)
if diff := abs(tb.Rate() - rate); diff/rate <= rateMargin {
return tb
}
}
panic("cannot find suitable quantum for " + strconv.FormatFloat(rate, 'g', -1, 64))
}
// nextQuantum returns the next quantum to try after q.
// We grow the quantum exponentially, but slowly, so we
// get a good fit in the lower numbers.
func nextQuantum(q int64) int64 {
q1 := q * 11 / 10
if q1 == q {
q1++
}
return q1
}
// newBucketWithQuantum is similar to NewBucket, but allows
// the specification of the quantum size - quantum tokens
// are added every fillInterval. This is so that we can get accurate
// rates even when we want to add more than one token per ns.
func newBucketWithQuantum(fillInterval time.Duration, capacity, quantum int64) *Bucket {
if fillInterval <= 0 {
panic("token bucket fill interval is not > 0")
}
if capacity <= 0 {
panic("token bucket capacity is not > 0")
}
if quantum <= 0 {
panic("token bucket quantum is not > 0")
}
return &Bucket{
startTime: time.Now(),
capacity: capacity,
quantum: quantum,
avail: capacity,
fillInterval: fillInterval,
}
}
// Wait takes count tokens from the bucket, waiting until they are
// available.
func (tb *Bucket) Wait(count int64) {
if d := tb.Take(count); d > 0 {
time.Sleep(d)
}
}
// WaitMaxDuration is like Wait except that it will
// only take tokens from the bucket if it needs to wait
// for no greater than maxWait. It reports whether
// any tokens have been removed from the bucket
// If no tokens have been removed, it returns immediately.
func (tb *Bucket) WaitMaxDuration(count int64, maxWait time.Duration) bool {
d, ok := tb.TakeMaxDuration(count, maxWait)
if d > 0 {
time.Sleep(d)
}
return ok
}
const infinityDuration time.Duration = 0x7fffffffffffffff
// Take takes count tokens from the bucket without blocking. It returns
// the time that the caller should wait until the tokens are actually
// available.
//
// Note that if the request is irrevocable - there is no way to return
// tokens to the bucket once this method commits us to taking them.
func (tb *Bucket) Take(count int64) time.Duration {
d, _ := tb.take(time.Now(), count, infinityDuration)
return d
}
// TakeMaxDuration is like Take, except that
// it will only take tokens from the bucket if the wait
// time for the tokens is no greater than maxWait.
//
// If it would take longer than maxWait for the tokens
// to become available, it does nothing and reports false,
// otherwise it returns the time that the caller should
// wait until the tokens are actually available, and reports
// true.
func (tb *Bucket) TakeMaxDuration(count int64, maxWait time.Duration) (time.Duration, bool) {
return tb.take(time.Now(), count, maxWait)
}
// TakeAvailable takes up to count immediately available tokens from the
// bucket. It returns the number of tokens removed, or zero if there are
// no available tokens. It does not block.
func (tb *Bucket) TakeAvailable(count int64) int64 {
return tb.takeAvailable(time.Now(), count)
}
// takeAvailable is the internal version of TakeAvailable - it takes the
// current time as an argument to enable easy testing.
func (tb *Bucket) takeAvailable(now time.Time, count int64) int64 {
if count <= 0 {
return 0
}
tb.mu.Lock()
defer tb.mu.Unlock()
tb.adjust(now)
if tb.avail <= 0 {
return 0
}
if count > tb.avail {
count = tb.avail
}
tb.avail -= count
return count
}
// Rate returns the fill rate of the bucket, in tokens per second.
func (tb *Bucket) Rate() float64 {
return 1e9 * float64(tb.quantum) / float64(tb.fillInterval)
}
// take is the internal version of Take - it takes the current time as
// an argument to enable easy testing.
func (tb *Bucket) take(now time.Time, count int64, maxWait time.Duration) (time.Duration, bool) {
if count <= 0 {
return 0, true
}
tb.mu.Lock()
defer tb.mu.Unlock()
currentTick := tb.adjust(now)
avail := tb.avail - count
if avail >= 0 {
tb.avail = avail
return 0, true
}
// Round up the missing tokens to the nearest multiple
// of quantum - the tokens won't be available until
// that tick.
endTick := currentTick + (-avail+tb.quantum-1)/tb.quantum
endTime := tb.startTime.Add(time.Duration(endTick) * tb.fillInterval)
waitTime := endTime.Sub(now)
if waitTime > maxWait {
return 0, false
}
tb.avail = avail
return waitTime, true
}
// adjust adjusts the current bucket capacity based on the current time.
// It returns the current tick.
func (tb *Bucket) adjust(now time.Time) (currentTick int64) {
currentTick = int64(now.Sub(tb.startTime) / tb.fillInterval)
if tb.avail >= tb.capacity {
return
}
tb.avail += (currentTick - tb.availTick) * tb.quantum
if tb.avail > tb.capacity {
tb.avail = tb.capacity
}
tb.availTick = currentTick
return
}
func abs(f float64) float64 {
if f < 0 {
return -f
}
return f
}

View File

@@ -0,0 +1,328 @@
// Copyright 2014 Canonical Ltd.
// Licensed under the LGPLv3 with static-linking exception.
// See LICENCE file for details.
package ratelimit
import (
gc "launchpad.net/gocheck"
"testing"
"time"
)
func TestPackage(t *testing.T) {
gc.TestingT(t)
}
type rateLimitSuite struct{}
var _ = gc.Suite(rateLimitSuite{})
type takeReq struct {
time time.Duration
count int64
expectWait time.Duration
}
var takeTests = []struct {
about string
fillInterval time.Duration
capacity int64
reqs []takeReq
}{{
about: "serial requests",
fillInterval: 250 * time.Millisecond,
capacity: 10,
reqs: []takeReq{{
time: 0,
count: 0,
expectWait: 0,
}, {
time: 0,
count: 10,
expectWait: 0,
}, {
time: 0,
count: 1,
expectWait: 250 * time.Millisecond,
}, {
time: 250 * time.Millisecond,
count: 1,
expectWait: 250 * time.Millisecond,
}},
}, {
about: "concurrent requests",
fillInterval: 250 * time.Millisecond,
capacity: 10,
reqs: []takeReq{{
time: 0,
count: 10,
expectWait: 0,
}, {
time: 0,
count: 2,
expectWait: 500 * time.Millisecond,
}, {
time: 0,
count: 2,
expectWait: 1000 * time.Millisecond,
}, {
time: 0,
count: 1,
expectWait: 1250 * time.Millisecond,
}},
}, {
about: "more than capacity",
fillInterval: 1 * time.Millisecond,
capacity: 10,
reqs: []takeReq{{
time: 0,
count: 10,
expectWait: 0,
}, {
time: 20 * time.Millisecond,
count: 15,
expectWait: 5 * time.Millisecond,
}},
}, {
about: "sub-quantum time",
fillInterval: 10 * time.Millisecond,
capacity: 10,
reqs: []takeReq{{
time: 0,
count: 10,
expectWait: 0,
}, {
time: 7 * time.Millisecond,
count: 1,
expectWait: 3 * time.Millisecond,
}, {
time: 8 * time.Millisecond,
count: 1,
expectWait: 12 * time.Millisecond,
}},
}, {
about: "within capacity",
fillInterval: 10 * time.Millisecond,
capacity: 5,
reqs: []takeReq{{
time: 0,
count: 5,
expectWait: 0,
}, {
time: 60 * time.Millisecond,
count: 5,
expectWait: 0,
}, {
time: 60 * time.Millisecond,
count: 1,
expectWait: 10 * time.Millisecond,
}, {
time: 80 * time.Millisecond,
count: 2,
expectWait: 10 * time.Millisecond,
}},
}}
func (rateLimitSuite) TestTake(c *gc.C) {
for i, test := range takeTests {
tb := NewBucket(test.fillInterval, test.capacity)
for j, req := range test.reqs {
d, ok := tb.take(tb.startTime.Add(req.time), req.count, infinityDuration)
c.Assert(ok, gc.Equals, true)
if d != req.expectWait {
c.Fatalf("test %d.%d, %s, got %v want %v", i, j, test.about, d, req.expectWait)
}
}
}
}
func (rateLimitSuite) TestTakeMaxDuration(c *gc.C) {
for i, test := range takeTests {
tb := NewBucket(test.fillInterval, test.capacity)
for j, req := range test.reqs {
if req.expectWait > 0 {
d, ok := tb.take(tb.startTime.Add(req.time), req.count, req.expectWait-1)
c.Assert(ok, gc.Equals, false)
c.Assert(d, gc.Equals, time.Duration(0))
}
d, ok := tb.take(tb.startTime.Add(req.time), req.count, req.expectWait)
c.Assert(ok, gc.Equals, true)
if d != req.expectWait {
c.Fatalf("test %d.%d, %s, got %v want %v", i, j, test.about, d, req.expectWait)
}
}
}
}
type takeAvailableReq struct {
time time.Duration
count int64
expect int64
}
var takeAvailableTests = []struct {
about string
fillInterval time.Duration
capacity int64
reqs []takeAvailableReq
}{{
about: "serial requests",
fillInterval: 250 * time.Millisecond,
capacity: 10,
reqs: []takeAvailableReq{{
time: 0,
count: 0,
expect: 0,
}, {
time: 0,
count: 10,
expect: 10,
}, {
time: 0,
count: 1,
expect: 0,
}, {
time: 250 * time.Millisecond,
count: 1,
expect: 1,
}},
}, {
about: "concurrent requests",
fillInterval: 250 * time.Millisecond,
capacity: 10,
reqs: []takeAvailableReq{{
time: 0,
count: 5,
expect: 5,
}, {
time: 0,
count: 2,
expect: 2,
}, {
time: 0,
count: 5,
expect: 3,
}, {
time: 0,
count: 1,
expect: 0,
}},
}, {
about: "more than capacity",
fillInterval: 1 * time.Millisecond,
capacity: 10,
reqs: []takeAvailableReq{{
time: 0,
count: 10,
expect: 10,
}, {
time: 20 * time.Millisecond,
count: 15,
expect: 10,
}},
}, {
about: "within capacity",
fillInterval: 10 * time.Millisecond,
capacity: 5,
reqs: []takeAvailableReq{{
time: 0,
count: 5,
expect: 5,
}, {
time: 60 * time.Millisecond,
count: 5,
expect: 5,
}, {
time: 70 * time.Millisecond,
count: 1,
expect: 1,
}},
}}
func (rateLimitSuite) TestTakeAvailable(c *gc.C) {
for i, test := range takeAvailableTests {
tb := NewBucket(test.fillInterval, test.capacity)
for j, req := range test.reqs {
d := tb.takeAvailable(tb.startTime.Add(req.time), req.count)
if d != req.expect {
c.Fatalf("test %d.%d, %s, got %v want %v", i, j, test.about, d, req.expect)
}
}
}
}
func (rateLimitSuite) TestPanics(c *gc.C) {
c.Assert(func() { NewBucket(0, 1) }, gc.PanicMatches, "token bucket fill interval is not > 0")
c.Assert(func() { NewBucket(-2, 1) }, gc.PanicMatches, "token bucket fill interval is not > 0")
c.Assert(func() { NewBucket(1, 0) }, gc.PanicMatches, "token bucket capacity is not > 0")
c.Assert(func() { NewBucket(1, -2) }, gc.PanicMatches, "token bucket capacity is not > 0")
}
func isCloseTo(x, y, tolerance float64) bool {
return abs(x-y)/y < tolerance
}
func (rateLimitSuite) TestRate(c *gc.C) {
tb := NewBucket(1, 1)
if !isCloseTo(tb.Rate(), 1e9, 0.00001) {
c.Fatalf("got %v want 1e9", tb.Rate())
}
tb = NewBucket(2*time.Second, 1)
if !isCloseTo(tb.Rate(), 0.5, 0.00001) {
c.Fatalf("got %v want 0.5", tb.Rate())
}
tb = newBucketWithQuantum(100*time.Millisecond, 1, 5)
if !isCloseTo(tb.Rate(), 50, 0.00001) {
c.Fatalf("got %v want 50", tb.Rate())
}
}
func checkRate(c *gc.C, rate float64) {
tb := NewBucketWithRate(rate, 1<<62)
if !isCloseTo(tb.Rate(), rate, rateMargin) {
c.Fatalf("got %g want %v", tb.Rate(), rate)
}
d, ok := tb.take(tb.startTime, 1<<62, infinityDuration)
c.Assert(ok, gc.Equals, true)
c.Assert(d, gc.Equals, time.Duration(0))
// Check that the actual rate is as expected by
// asking for a not-quite multiple of the bucket's
// quantum and checking that the wait time
// correct.
d, ok = tb.take(tb.startTime, tb.quantum*2-tb.quantum/2, infinityDuration)
c.Assert(ok, gc.Equals, true)
expectTime := 1e9 * float64(tb.quantum) * 2 / rate
if !isCloseTo(float64(d), expectTime, rateMargin) {
c.Fatalf("rate %g: got %g want %v", rate, float64(d), expectTime)
}
}
func (rateLimitSuite) TestNewWithRate(c *gc.C) {
for rate := float64(1); rate < 1e6; rate += 7 {
checkRate(c, rate)
}
for _, rate := range []float64{
1024 * 1024 * 1024,
1e-5,
0.9e-5,
0.5,
0.9,
0.9e8,
3e12,
4e18,
} {
checkRate(c, rate)
checkRate(c, rate/3)
checkRate(c, rate*1.3)
}
}
func BenchmarkWait(b *testing.B) {
tb := NewBucket(1, 16*1024)
for i := b.N - 1; i >= 0; i-- {
tb.Wait(1)
}
}

View File

@@ -0,0 +1,51 @@
// Copyright 2014 Canonical Ltd.
// Licensed under the LGPLv3 with static-linking exception.
// See LICENCE file for details.
package ratelimit
import "io"
type reader struct {
r io.Reader
bucket *Bucket
}
// Reader returns a reader that is rate limited by
// the given token bucket. Each token in the bucket
// represents one byte.
func Reader(r io.Reader, bucket *Bucket) io.Reader {
return &reader{
r: r,
bucket: bucket,
}
}
func (r *reader) Read(buf []byte) (int, error) {
n, err := r.r.Read(buf)
if n <= 0 {
return n, err
}
r.bucket.Wait(int64(n))
return n, err
}
type writer struct {
w io.Writer
bucket *Bucket
}
// Writer returns a reader that is rate limited by
// the given token bucket. Each token in the bucket
// represents one byte.
func Writer(w io.Writer, bucket *Bucket) io.Writer {
return &writer{
w: w,
bucket: bucket,
}
}
func (w *writer) Write(buf []byte) (int, error) {
w.bucket.Wait(int64(len(buf)))
return w.w.Write(buf)
}

View File

@@ -0,0 +1,815 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package coding implements low-level QR coding details.
package coding
import (
"fmt"
"strconv"
"strings"
"github.com/vitrun/qart/gf256"
)
// Field is the field for QR error correction.
var Field = gf256.NewField(0x11d, 2)
// A Version represents a QR version.
// The version specifies the size of the QR code:
// a QR code with version v has 4v+17 pixels on a side.
// Versions number from 1 to 40: the larger the version,
// the more information the code can store.
type Version int
const MinVersion = 1
const MaxVersion = 40
func (v Version) String() string {
return strconv.Itoa(int(v))
}
func (v Version) sizeClass() int {
if v <= 9 {
return 0
}
if v <= 26 {
return 1
}
return 2
}
// DataBytes returns the number of data bytes that can be
// stored in a QR code with the given version and level.
func (v Version) DataBytes(l Level) int {
vt := &vtab[v]
lev := &vt.level[l]
return vt.bytes - lev.nblock*lev.check
}
// Encoding implements a QR data encoding scheme.
// The implementations--Numeric, Alphanumeric, and String--specify
// the character set and the mapping from UTF-8 to code bits.
// The more restrictive the mode, the fewer code bits are needed.
type Encoding interface {
Check() error
Bits(v Version) int
Encode(b *Bits, v Version)
}
type Bits struct {
b []byte
nbit int
}
func (b *Bits) Reset() {
b.b = b.b[:0]
b.nbit = 0
}
func (b *Bits) Bits() int {
return b.nbit
}
func (b *Bits) Bytes() []byte {
if b.nbit%8 != 0 {
panic("fractional byte")
}
return b.b
}
func (b *Bits) Append(p []byte) {
if b.nbit%8 != 0 {
panic("fractional byte")
}
b.b = append(b.b, p...)
b.nbit += 8 * len(p)
}
func (b *Bits) Write(v uint, nbit int) {
for nbit > 0 {
n := nbit
if n > 8 {
n = 8
}
if b.nbit%8 == 0 {
b.b = append(b.b, 0)
} else {
m := -b.nbit & 7
if n > m {
n = m
}
}
b.nbit += n
sh := uint(nbit - n)
b.b[len(b.b)-1] |= uint8(v >> sh << uint(-b.nbit&7))
v -= v >> sh << sh
nbit -= n
}
}
// Num is the encoding for numeric data.
// The only valid characters are the decimal digits 0 through 9.
type Num string
func (s Num) String() string {
return fmt.Sprintf("Num(%#q)", string(s))
}
func (s Num) Check() error {
for _, c := range s {
if c < '0' || '9' < c {
return fmt.Errorf("non-numeric string %#q", string(s))
}
}
return nil
}
var numLen = [3]int{10, 12, 14}
func (s Num) Bits(v Version) int {
return 4 + numLen[v.sizeClass()] + (10*len(s)+2)/3
}
func (s Num) Encode(b *Bits, v Version) {
b.Write((uint)(1), 4)
b.Write(uint(len(s)), numLen[v.sizeClass()])
var i int
for i = 0; i+3 <= len(s); i += 3 {
w := uint(s[i]-'0')*100 + uint(s[i+1]-'0')*10 + uint(s[i+2]-'0')
b.Write(w, 10)
}
switch len(s) - i {
case 1:
w := uint(s[i] - '0')
b.Write(w, 4)
case 2:
w := uint(s[i]-'0')*10 + uint(s[i+1]-'0')
b.Write(w, 7)
}
}
// Alpha is the encoding for alphanumeric data.
// The valid characters are 0-9A-Z$%*+-./: and space.
type Alpha string
const alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ $%*+-./:"
func (s Alpha) String() string {
return fmt.Sprintf("Alpha(%#q)", string(s))
}
func (s Alpha) Check() error {
for _, c := range s {
if strings.IndexRune(alphabet, c) < 0 {
return fmt.Errorf("non-alphanumeric string %#q", string(s))
}
}
return nil
}
var alphaLen = [3]int{9, 11, 13}
func (s Alpha) Bits(v Version) int {
return 4 + alphaLen[v.sizeClass()] + (11*len(s)+1)/2
}
func (s Alpha) Encode(b *Bits, v Version) {
b.Write((uint)(2), 4)
b.Write(uint(len(s)), alphaLen[v.sizeClass()])
var i int
for i = 0; i+2 <= len(s); i += 2 {
w := uint(strings.IndexRune(alphabet, rune(s[i])))*45 +
uint(strings.IndexRune(alphabet, rune(s[i+1])))
b.Write(w, 11)
}
if i < len(s) {
w := uint(strings.IndexRune(alphabet, rune(s[i])))
b.Write(w, 6)
}
}
// String is the encoding for 8-bit data. All bytes are valid.
type String string
func (s String) String() string {
return fmt.Sprintf("String(%#q)", string(s))
}
func (s String) Check() error {
return nil
}
var stringLen = [3]int{8, 16, 16}
func (s String) Bits(v Version) int {
return 4 + stringLen[v.sizeClass()] + 8*len(s)
}
func (s String) Encode(b *Bits, v Version) {
b.Write((uint)(4), 4)
b.Write(uint(len(s)), stringLen[v.sizeClass()])
for i := 0; i < len(s); i++ {
b.Write(uint(s[i]), 8)
}
}
// A Pixel describes a single pixel in a QR code.
type Pixel uint32
const (
Black Pixel = 1 << iota
Invert
)
func (p Pixel) Offset() uint {
return uint(p >> 6)
}
func OffsetPixel(o uint) Pixel {
return Pixel(o << 6)
}
func (r PixelRole) Pixel() Pixel {
return Pixel(r << 2)
}
func (p Pixel) Role() PixelRole {
return PixelRole(p>>2) & 15
}
func (p Pixel) String() string {
s := p.Role().String()
if p&Black != 0 {
s += "+black"
}
if p&Invert != 0 {
s += "+invert"
}
s += "+" + strconv.FormatUint(uint64(p.Offset()), 10)
return s
}
// A PixelRole describes the role of a QR pixel.
type PixelRole uint32
const (
_ PixelRole = iota
Position // position squares (large)
Alignment // alignment squares (small)
Timing // timing strip between position squares
Format // format metadata
PVersion // version pattern
Unused // unused pixel
Data // data bit
Check // error correction check bit
Extra
)
var roles = []string{
"",
"position",
"alignment",
"timing",
"format",
"pversion",
"unused",
"data",
"check",
"extra",
}
func (r PixelRole) String() string {
if Position <= r && r <= Check {
return roles[r]
}
return strconv.Itoa(int(r))
}
// A Level represents a QR error correction level.
// From least to most tolerant of errors, they are L, M, Q, H.
type Level int
const (
L Level = iota
M
Q
H
)
func (l Level) String() string {
if L <= l && l <= H {
return "LMQH"[l : l+1]
}
return strconv.Itoa(int(l))
}
// A Code is a square pixel grid.
type Code struct {
Bitmap []byte // 1 is black, 0 is white
Size int // number of pixels on a side
Stride int // number of bytes per row
}
func (c *Code) Black(x, y int) bool {
return 0 <= x && x < c.Size && 0 <= y && y < c.Size &&
c.Bitmap[y*c.Stride+x/8]&(1<<uint(7-x&7)) != 0
}
// A Mask describes a mask that is applied to the QR
// code to avoid QR artifacts being interpreted as
// alignment and timing patterns (such as the squares
// in the corners). Valid masks are integers from 0 to 7.
type Mask int
// http://www.swetake.com/qr/qr5_en.html
var mfunc = []func(int, int) bool{
func(i, j int) bool { return (i+j)%2 == 0 },
func(i, _ int) bool { return i%2 == 0 },
func(_, j int) bool { return j%3 == 0 },
func(i, j int) bool { return (i+j)%3 == 0 },
func(i, j int) bool { return (i/2+j/3)%2 == 0 },
func(i, j int) bool { return i*j%2+i*j%3 == 0 },
func(i, j int) bool { return (i*j%2+i*j%3)%2 == 0 },
func(i, j int) bool { return (i*j%3+(i+j)%2)%2 == 0 },
}
func (m Mask) Invert(y, x int) bool {
if m < 0 {
return false
}
return mfunc[m](y, x)
}
// A Plan describes how to construct a QR code
// with a specific version, level, and mask.
type Plan struct {
Version Version
Level Level
Mask Mask
DataBytes int // number of data bytes
CheckBytes int // number of error correcting (checksum) bytes
Blocks int // number of data blocks
Pixel [][]Pixel // pixel map
}
// NewPlan returns a Plan for a QR code with the given
// version, level, and mask.
func NewPlan(version Version, level Level, mask Mask) (*Plan, error) {
p, err := vplan(version)
if err != nil {
return nil, err
}
if err := fplan(level, mask, p); err != nil {
return nil, err
}
if err := lplan(version, level, p); err != nil {
return nil, err
}
if err := mplan(mask, p); err != nil {
return nil, err
}
return p, nil
}
func (b *Bits) Pad(n int) {
if n < 0 {
panic("qr: invalid pad size")
}
if n <= 4 {
b.Write((uint)(0), n)
} else {
b.Write((uint)(0), 4)
n -= 4
n -= -b.Bits() & 7
b.Write((uint)(0), -b.Bits()&7)
pad := n / 8
for i := 0; i < pad; i += 2 {
b.Write((uint)(0xec), 8)
if i+1 >= pad {
break
}
b.Write((uint)(0x11), 8)
}
}
}
func (b *Bits) AddCheckBytes(v Version, l Level) {
nd := v.DataBytes(l)
if b.nbit < nd*8 {
b.Pad(nd*8 - b.nbit)
}
if b.nbit != nd*8 {
panic("qr: too much data")
}
dat := b.Bytes()
vt := &vtab[v]
lev := &vt.level[l]
db := nd / lev.nblock
extra := nd % lev.nblock
chk := make([]byte, lev.check)
rs := gf256.NewRSEncoder(Field, lev.check)
for i := 0; i < lev.nblock; i++ {
if i == lev.nblock-extra {
db++
}
rs.ECC(dat[:db], chk)
b.Append(chk)
dat = dat[db:]
}
if len(b.Bytes()) != vt.bytes {
panic("qr: internal error")
}
}
func (p *Plan) Encode(text ...Encoding) (*Code, error) {
var b Bits
for _, t := range text {
if err := t.Check(); err != nil {
return nil, err
}
t.Encode(&b, p.Version)
}
if b.Bits() > p.DataBytes*8 {
return nil, fmt.Errorf("cannot encode %d bits into %d-bit code", b.Bits(), p.DataBytes*8)
}
b.AddCheckBytes(p.Version, p.Level)
bytes := b.Bytes()
// Now we have the checksum bytes and the data bytes.
// Construct the actual code.
c := &Code{Size: len(p.Pixel), Stride: (len(p.Pixel) + 7) &^ 7}
c.Bitmap = make([]byte, c.Stride*c.Size)
crow := c.Bitmap
for _, row := range p.Pixel {
for x, pix := range row {
switch pix.Role() {
case Data, Check:
o := pix.Offset()
if bytes[o/8]&(1<<uint(7-o&7)) != 0 {
pix ^= Black
}
}
if pix&Black != 0 {
crow[x/8] |= 1 << uint(7-x&7)
}
}
crow = crow[c.Stride:]
}
return c, nil
}
// A version describes metadata associated with a version.
type version struct {
apos int
astride int
bytes int
pattern int
level [4]level
}
type level struct {
nblock int
check int
}
var vtab = []version{
{},
{100, 100, 26, 0x0, [4]level{{1, 7}, {1, 10}, {1, 13}, {1, 17}}}, // 1
{16, 100, 44, 0x0, [4]level{{1, 10}, {1, 16}, {1, 22}, {1, 28}}}, // 2
{20, 100, 70, 0x0, [4]level{{1, 15}, {1, 26}, {2, 18}, {2, 22}}}, // 3
{24, 100, 100, 0x0, [4]level{{1, 20}, {2, 18}, {2, 26}, {4, 16}}}, // 4
{28, 100, 134, 0x0, [4]level{{1, 26}, {2, 24}, {4, 18}, {4, 22}}}, // 5
{32, 100, 172, 0x0, [4]level{{2, 18}, {4, 16}, {4, 24}, {4, 28}}}, // 6
{20, 16, 196, 0x7c94, [4]level{{2, 20}, {4, 18}, {6, 18}, {5, 26}}}, // 7
{22, 18, 242, 0x85bc, [4]level{{2, 24}, {4, 22}, {6, 22}, {6, 26}}}, // 8
{24, 20, 292, 0x9a99, [4]level{{2, 30}, {5, 22}, {8, 20}, {8, 24}}}, // 9
{26, 22, 346, 0xa4d3, [4]level{{4, 18}, {5, 26}, {8, 24}, {8, 28}}}, // 10
{28, 24, 404, 0xbbf6, [4]level{{4, 20}, {5, 30}, {8, 28}, {11, 24}}}, // 11
{30, 26, 466, 0xc762, [4]level{{4, 24}, {8, 22}, {10, 26}, {11, 28}}}, // 12
{32, 28, 532, 0xd847, [4]level{{4, 26}, {9, 22}, {12, 24}, {16, 22}}}, // 13
{24, 20, 581, 0xe60d, [4]level{{4, 30}, {9, 24}, {16, 20}, {16, 24}}}, // 14
{24, 22, 655, 0xf928, [4]level{{6, 22}, {10, 24}, {12, 30}, {18, 24}}}, // 15
{24, 24, 733, 0x10b78, [4]level{{6, 24}, {10, 28}, {17, 24}, {16, 30}}}, // 16
{28, 24, 815, 0x1145d, [4]level{{6, 28}, {11, 28}, {16, 28}, {19, 28}}}, // 17
{28, 26, 901, 0x12a17, [4]level{{6, 30}, {13, 26}, {18, 28}, {21, 28}}}, // 18
{28, 28, 991, 0x13532, [4]level{{7, 28}, {14, 26}, {21, 26}, {25, 26}}}, // 19
{32, 28, 1085, 0x149a6, [4]level{{8, 28}, {16, 26}, {20, 30}, {25, 28}}}, // 20
{26, 22, 1156, 0x15683, [4]level{{8, 28}, {17, 26}, {23, 28}, {25, 30}}}, // 21
{24, 24, 1258, 0x168c9, [4]level{{9, 28}, {17, 28}, {23, 30}, {34, 24}}}, // 22
{28, 24, 1364, 0x177ec, [4]level{{9, 30}, {18, 28}, {25, 30}, {30, 30}}}, // 23
{26, 26, 1474, 0x18ec4, [4]level{{10, 30}, {20, 28}, {27, 30}, {32, 30}}}, // 24
{30, 26, 1588, 0x191e1, [4]level{{12, 26}, {21, 28}, {29, 30}, {35, 30}}}, // 25
{28, 28, 1706, 0x1afab, [4]level{{12, 28}, {23, 28}, {34, 28}, {37, 30}}}, // 26
{32, 28, 1828, 0x1b08e, [4]level{{12, 30}, {25, 28}, {34, 30}, {40, 30}}}, // 27
{24, 24, 1921, 0x1cc1a, [4]level{{13, 30}, {26, 28}, {35, 30}, {42, 30}}}, // 28
{28, 24, 2051, 0x1d33f, [4]level{{14, 30}, {28, 28}, {38, 30}, {45, 30}}}, // 29
{24, 26, 2185, 0x1ed75, [4]level{{15, 30}, {29, 28}, {40, 30}, {48, 30}}}, // 30
{28, 26, 2323, 0x1f250, [4]level{{16, 30}, {31, 28}, {43, 30}, {51, 30}}}, // 31
{32, 26, 2465, 0x209d5, [4]level{{17, 30}, {33, 28}, {45, 30}, {54, 30}}}, // 32
{28, 28, 2611, 0x216f0, [4]level{{18, 30}, {35, 28}, {48, 30}, {57, 30}}}, // 33
{32, 28, 2761, 0x228ba, [4]level{{19, 30}, {37, 28}, {51, 30}, {60, 30}}}, // 34
{28, 24, 2876, 0x2379f, [4]level{{19, 30}, {38, 28}, {53, 30}, {63, 30}}}, // 35
{22, 26, 3034, 0x24b0b, [4]level{{20, 30}, {40, 28}, {56, 30}, {66, 30}}}, // 36
{26, 26, 3196, 0x2542e, [4]level{{21, 30}, {43, 28}, {59, 30}, {70, 30}}}, // 37
{30, 26, 3362, 0x26a64, [4]level{{22, 30}, {45, 28}, {62, 30}, {74, 30}}}, // 38
{24, 28, 3532, 0x27541, [4]level{{24, 30}, {47, 28}, {65, 30}, {77, 30}}}, // 39
{28, 28, 3706, 0x28c69, [4]level{{25, 30}, {49, 28}, {68, 30}, {81, 30}}}, // 40
}
func grid(siz int) [][]Pixel {
m := make([][]Pixel, siz)
pix := make([]Pixel, siz*siz)
for i := range m {
m[i], pix = pix[:siz], pix[siz:]
}
return m
}
// vplan creates a Plan for the given version.
func vplan(v Version) (*Plan, error) {
p := &Plan{Version: v}
if v < 1 || v > 40 {
return nil, fmt.Errorf("invalid QR version %d", int(v))
}
siz := 17 + int(v)*4
m := grid(siz)
p.Pixel = m
// Timing markers (overwritten by boxes).
const ti = 6 // timing is in row/column 6 (counting from 0)
for i := range m {
p := Timing.Pixel()
if i&1 == 0 {
p |= Black
}
m[i][ti] = p
m[ti][i] = p
}
// Position boxes.
posBox(m, 0, 0)
posBox(m, siz-7, 0)
posBox(m, 0, siz-7)
// Alignment boxes.
info := &vtab[v]
for x := 4; x+5 < siz; {
for y := 4; y+5 < siz; {
// don't overwrite timing markers
if (x < 7 && y < 7) || (x < 7 && y+5 >= siz-7) || (x+5 >= siz-7 && y < 7) {
} else {
alignBox(m, x, y)
}
if y == 4 {
y = info.apos
} else {
y += info.astride
}
}
if x == 4 {
x = info.apos
} else {
x += info.astride
}
}
// Version pattern.
pat := vtab[v].pattern
if pat != 0 {
v := pat
for x := 0; x < 6; x++ {
for y := 0; y < 3; y++ {
p := PVersion.Pixel()
if v&1 != 0 {
p |= Black
}
m[siz-11+y][x] = p
m[x][siz-11+y] = p
v >>= 1
}
}
}
// One lonely black pixel
m[siz-8][8] = Unused.Pixel() | Black
return p, nil
}
// fplan adds the format pixels
func fplan(l Level, m Mask, p *Plan) error {
// Format pixels.
fb := uint32(l^1) << 13 // level: L=01, M=00, Q=11, H=10
fb |= uint32(m) << 10 // mask
const formatPoly = 0x537
rem := fb
for i := 14; i >= 10; i-- {
if rem&(1<<uint(i)) != 0 {
rem ^= formatPoly << uint(i-10)
}
}
fb |= rem
invert := uint32(0x5412)
siz := len(p.Pixel)
for i := uint(0); i < 15; i++ {
pix := Format.Pixel() + OffsetPixel(i)
if (fb>>i)&1 == 1 {
pix |= Black
}
if (invert>>i)&1 == 1 {
pix ^= Invert | Black
}
// top left
switch {
case i < 6:
p.Pixel[i][8] = pix
case i < 8:
p.Pixel[i+1][8] = pix
case i < 9:
p.Pixel[8][7] = pix
default:
p.Pixel[8][14-i] = pix
}
// bottom right
switch {
case i < 8:
p.Pixel[8][siz-1-int(i)] = pix
default:
p.Pixel[siz-1-int(14-i)][8] = pix
}
}
return nil
}
// lplan edits a version-only Plan to add information
// about the error correction levels.
func lplan(v Version, l Level, p *Plan) error {
p.Level = l
nblock := vtab[v].level[l].nblock
ne := vtab[v].level[l].check
nde := (vtab[v].bytes - ne*nblock) / nblock
extra := (vtab[v].bytes - ne*nblock) % nblock
dataBits := (nde*nblock + extra) * 8
checkBits := ne * nblock * 8
p.DataBytes = vtab[v].bytes - ne*nblock
p.CheckBytes = ne * nblock
p.Blocks = nblock
// Make data + checksum pixels.
data := make([]Pixel, dataBits)
for i := range data {
data[i] = Data.Pixel() | OffsetPixel(uint(i))
}
check := make([]Pixel, checkBits)
for i := range check {
check[i] = Check.Pixel() | OffsetPixel(uint(i+dataBits))
}
// Split into blocks.
dataList := make([][]Pixel, nblock)
checkList := make([][]Pixel, nblock)
for i := 0; i < nblock; i++ {
// The last few blocks have an extra data byte (8 pixels).
nd := nde
if i >= nblock-extra {
nd++
}
dataList[i], data = data[0:nd*8], data[nd*8:]
checkList[i], check = check[0:ne*8], check[ne*8:]
}
if len(data) != 0 || len(check) != 0 {
panic("data/check math")
}
// Build up bit sequence, taking first byte of each block,
// then second byte, and so on. Then checksums.
bits := make([]Pixel, dataBits+checkBits)
dst := bits
for i := 0; i < nde+1; i++ {
for _, b := range dataList {
if i*8 < len(b) {
copy(dst, b[i*8:(i+1)*8])
dst = dst[8:]
}
}
}
for i := 0; i < ne; i++ {
for _, b := range checkList {
if i*8 < len(b) {
copy(dst, b[i*8:(i+1)*8])
dst = dst[8:]
}
}
}
if len(dst) != 0 {
panic("dst math")
}
// Sweep up pair of columns,
// then down, assigning to right then left pixel.
// Repeat.
// See Figure 2 of http://www.pclviewer.com/rs2/qrtopology.htm
siz := len(p.Pixel)
rem := make([]Pixel, 7)
for i := range rem {
rem[i] = Extra.Pixel()
}
src := append(bits, rem...)
for x := siz; x > 0; {
for y := siz - 1; y >= 0; y-- {
if p.Pixel[y][x-1].Role() == 0 {
p.Pixel[y][x-1], src = src[0], src[1:]
}
if p.Pixel[y][x-2].Role() == 0 {
p.Pixel[y][x-2], src = src[0], src[1:]
}
}
x -= 2
if x == 7 { // vertical timing strip
x--
}
for y := 0; y < siz; y++ {
if p.Pixel[y][x-1].Role() == 0 {
p.Pixel[y][x-1], src = src[0], src[1:]
}
if p.Pixel[y][x-2].Role() == 0 {
p.Pixel[y][x-2], src = src[0], src[1:]
}
}
x -= 2
}
return nil
}
// mplan edits a version+level-only Plan to add the mask.
func mplan(m Mask, p *Plan) error {
p.Mask = m
for y, row := range p.Pixel {
for x, pix := range row {
if r := pix.Role(); (r == Data || r == Check || r == Extra) && p.Mask.Invert(y, x) {
row[x] ^= Black | Invert
}
}
}
return nil
}
// posBox draws a position (large) box at upper left x, y.
func posBox(m [][]Pixel, x, y int) {
pos := Position.Pixel()
// box
for dy := 0; dy < 7; dy++ {
for dx := 0; dx < 7; dx++ {
p := pos
if dx == 0 || dx == 6 || dy == 0 || dy == 6 || 2 <= dx && dx <= 4 && 2 <= dy && dy <= 4 {
p |= Black
}
m[y+dy][x+dx] = p
}
}
// white border
for dy := -1; dy < 8; dy++ {
if 0 <= y+dy && y+dy < len(m) {
if x > 0 {
m[y+dy][x-1] = pos
}
if x+7 < len(m) {
m[y+dy][x+7] = pos
}
}
}
for dx := -1; dx < 8; dx++ {
if 0 <= x+dx && x+dx < len(m) {
if y > 0 {
m[y-1][x+dx] = pos
}
if y+7 < len(m) {
m[y+7][x+dx] = pos
}
}
}
}
// alignBox draw an alignment (small) box at upper left x, y.
func alignBox(m [][]Pixel, x, y int) {
// box
align := Alignment.Pixel()
for dy := 0; dy < 5; dy++ {
for dx := 0; dx < 5; dx++ {
p := align
if dx == 0 || dx == 4 || dy == 0 || dy == 4 || dx == 2 && dy == 2 {
p |= Black
}
m[y+dy][x+dx] = p
}
}
}

View File

@@ -0,0 +1,241 @@
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Package gf256 implements arithmetic over the Galois Field GF(256).
package gf256
import "strconv"
// A Field represents an instance of GF(256) defined by a specific polynomial.
type Field struct {
log [256]byte // log[0] is unused
exp [510]byte
}
// NewField returns a new field corresponding to the polynomial poly
// and generator α. The Reed-Solomon encoding in QR codes uses
// polynomial 0x11d with generator 2.
//
// The choice of generator α only affects the Exp and Log operations.
func NewField(poly, α int) *Field {
if poly < 0x100 || poly >= 0x200 || reducible(poly) {
panic("gf256: invalid polynomial: " + strconv.Itoa(poly))
}
var f Field
x := 1
for i := 0; i < 255; i++ {
if x == 1 && i != 0 {
panic("gf256: invalid generator " + strconv.Itoa(α) +
" for polynomial " + strconv.Itoa(poly))
}
f.exp[i] = byte(x)
f.exp[i+255] = byte(x)
f.log[x] = byte(i)
x = mul(x, α, poly)
}
f.log[0] = 255
for i := 0; i < 255; i++ {
if f.log[f.exp[i]] != byte(i) {
panic("bad log")
}
if f.log[f.exp[i+255]] != byte(i) {
panic("bad log")
}
}
for i := 1; i < 256; i++ {
if f.exp[f.log[i]] != byte(i) {
panic("bad log")
}
}
return &f
}
// nbit returns the number of significant in p.
func nbit(p int) uint {
n := uint(0)
for ; p > 0; p >>= 1 {
n++
}
return n
}
// polyDiv divides the polynomial p by q and returns the remainder.
func polyDiv(p, q int) int {
np := nbit(p)
nq := nbit(q)
for ; np >= nq; np-- {
if p&(1<<(np-1)) != 0 {
p ^= q << (np - nq)
}
}
return p
}
// mul returns the product x*y mod poly, a GF(256) multiplication.
func mul(x, y, poly int) int {
z := 0
for x > 0 {
if x&1 != 0 {
z ^= y
}
x >>= 1
y <<= 1
if y&0x100 != 0 {
y ^= poly
}
}
return z
}
// reducible reports whether p is reducible.
func reducible(p int) bool {
// Multiplying n-bit * n-bit produces (2n-1)-bit,
// so if p is reducible, one of its factors must be
// of np/2+1 bits or fewer.
np := nbit(p)
for q := 2; q < int(1<<(np/2+1)); q++ {
if polyDiv(p, q) == 0 {
return true
}
}
return false
}
// Add returns the sum of x and y in the field.
func (f *Field) Add(x, y byte) byte {
return x ^ y
}
// Exp returns the the base-α exponential of e in the field.
// If e < 0, Exp returns 0.
func (f *Field) Exp(e int) byte {
if e < 0 {
return 0
}
return f.exp[e%255]
}
// Log returns the base-α logarithm of x in the field.
// If x == 0, Log returns -1.
func (f *Field) Log(x byte) int {
if x == 0 {
return -1
}
return int(f.log[x])
}
// Inv returns the multiplicative inverse of x in the field.
// If x == 0, Inv returns 0.
func (f *Field) Inv(x byte) byte {
if x == 0 {
return 0
}
return f.exp[255-f.log[x]]
}
// Mul returns the product of x and y in the field.
func (f *Field) Mul(x, y byte) byte {
if x == 0 || y == 0 {
return 0
}
return f.exp[int(f.log[x])+int(f.log[y])]
}
// An RSEncoder implements Reed-Solomon encoding
// over a given field using a given number of error correction bytes.
type RSEncoder struct {
f *Field
c int
gen []byte
lgen []byte
p []byte
}
func (f *Field) gen(e int) (gen, lgen []byte) {
// p = 1
p := make([]byte, e+1)
p[e] = 1
for i := 0; i < e; i++ {
// p *= (x + Exp(i))
// p[j] = p[j]*Exp(i) + p[j+1].
c := f.Exp(i)
for j := 0; j < e; j++ {
p[j] = f.Mul(p[j], c) ^ p[j+1]
}
p[e] = f.Mul(p[e], c)
}
// lp = log p.
lp := make([]byte, e+1)
for i, c := range p {
if c == 0 {
lp[i] = 255
} else {
lp[i] = byte(f.Log(c))
}
}
return p, lp
}
// NewRSEncoder returns a new Reed-Solomon encoder
// over the given field and number of error correction bytes.
func NewRSEncoder(f *Field, c int) *RSEncoder {
gen, lgen := f.gen(c)
return &RSEncoder{f: f, c: c, gen: gen, lgen: lgen}
}
// ECC writes to check the error correcting code bytes
// for data using the given Reed-Solomon parameters.
func (rs *RSEncoder) ECC(data []byte, check []byte) {
if len(check) < rs.c {
panic("gf256: invalid check byte length")
}
if rs.c == 0 {
return
}
// The check bytes are the remainder after dividing
// data padded with c zeros by the generator polynomial.
// p = data padded with c zeros.
var p []byte
n := len(data) + rs.c
if len(rs.p) >= n {
p = rs.p
} else {
p = make([]byte, n)
}
copy(p, data)
for i := len(data); i < len(p); i++ {
p[i] = 0
}
// Divide p by gen, leaving the remainder in p[len(data):].
// p[0] is the most significant term in p, and
// gen[0] is the most significant term in the generator,
// which is always 1.
// To avoid repeated work, we store various values as
// lv, not v, where lv = log[v].
f := rs.f
lgen := rs.lgen[1:]
for i := 0; i < len(data); i++ {
c := p[i]
if c == 0 {
continue
}
q := p[i+1:]
exp := f.exp[f.log[c]:]
for j, lg := range lgen {
if lg != 255 { // lgen uses 255 for log 0
q[j] ^= exp[lg]
}
}
}
copy(check, p[len(data):])
rs.p = p
}

401
Godeps/_workspace/src/github.com/vitrun/qart/qr/png.go generated vendored Normal file
View File

@@ -0,0 +1,401 @@
// Copyright 2011 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
package qr
// PNG writer for QR codes.
import (
"bytes"
"encoding/binary"
"hash"
"hash/crc32"
)
// PNG returns a PNG image displaying the code.
//
// PNG uses a custom encoder tailored to QR codes.
// Its compressed size is about 2x away from optimal,
// but it runs about 20x faster than calling png.Encode
// on c.Image().
func (c *Code) PNG() []byte {
var p pngWriter
return p.encode(c)
}
type pngWriter struct {
tmp [16]byte
wctmp [4]byte
buf bytes.Buffer
zlib bitWriter
crc hash.Hash32
}
var pngHeader = []byte("\x89PNG\r\n\x1a\n")
func (w *pngWriter) encode(c *Code) []byte {
scale := c.Scale
siz := c.Size
w.buf.Reset()
// Header
w.buf.Write(pngHeader)
// Header block
binary.BigEndian.PutUint32(w.tmp[0:4], uint32((siz+8)*scale))
binary.BigEndian.PutUint32(w.tmp[4:8], uint32((siz+8)*scale))
w.tmp[8] = 1 // 1-bit
w.tmp[9] = 0 // gray
w.tmp[10] = 0
w.tmp[11] = 0
w.tmp[12] = 0
w.writeChunk("IHDR", w.tmp[:13])
// Comment
w.writeChunk("tEXt", comment)
// Data
w.zlib.writeCode(c)
w.writeChunk("IDAT", w.zlib.bytes.Bytes())
// End
w.writeChunk("IEND", nil)
return w.buf.Bytes()
}
var comment = []byte("Software\x00QR-PNG http://qr.swtch.com/")
func (w *pngWriter) writeChunk(name string, data []byte) {
if w.crc == nil {
w.crc = crc32.NewIEEE()
}
binary.BigEndian.PutUint32(w.wctmp[0:4], uint32(len(data)))
w.buf.Write(w.wctmp[0:4])
w.crc.Reset()
copy(w.wctmp[0:4], name)
w.buf.Write(w.wctmp[0:4])
w.crc.Write(w.wctmp[0:4])
w.buf.Write(data)
w.crc.Write(data)
crc := w.crc.Sum32()
binary.BigEndian.PutUint32(w.wctmp[0:4], crc)
w.buf.Write(w.wctmp[0:4])
}
func (b *bitWriter) writeCode(c *Code) {
const ftNone = 0
b.adler32.Reset()
b.bytes.Reset()
b.nbit = 0
scale := c.Scale
siz := c.Size
// zlib header
b.tmp[0] = 0x78
b.tmp[1] = 0
b.tmp[1] += uint8(31 - (uint16(b.tmp[0])<<8+uint16(b.tmp[1]))%31)
b.bytes.Write(b.tmp[0:2])
// Start flate block.
b.writeBits(1, 1, false) // final block
b.writeBits(1, 2, false) // compressed, fixed Huffman tables
// White border.
// First row.
b.byte(ftNone)
n := (scale*(siz+8) + 7) / 8
b.byte(255)
b.repeat(n-1, 1)
// 4*scale rows total.
b.repeat((4*scale-1)*(1+n), 1+n)
for i := 0; i < 4*scale; i++ {
b.adler32.WriteNByte(ftNone, 1)
b.adler32.WriteNByte(255, n)
}
row := make([]byte, 1+n)
for y := 0; y < siz; y++ {
row[0] = ftNone
j := 1
var z uint8
nz := 0
for x := -4; x < siz+4; x++ {
// Raw data.
for i := 0; i < scale; i++ {
z <<= 1
if !c.Black(x, y) {
z |= 1
}
if nz++; nz == 8 {
row[j] = z
j++
nz = 0
}
}
}
if j < len(row) {
row[j] = z
}
for _, z := range row {
b.byte(z)
}
// Scale-1 copies.
b.repeat((scale-1)*(1+n), 1+n)
b.adler32.WriteN(row, scale)
}
// White border.
// First row.
b.byte(ftNone)
b.byte(255)
b.repeat(n-1, 1)
// 4*scale rows total.
b.repeat((4*scale-1)*(1+n), 1+n)
for i := 0; i < 4*scale; i++ {
b.adler32.WriteNByte(ftNone, 1)
b.adler32.WriteNByte(255, n)
}
// End of block.
b.hcode(256)
b.flushBits()
// adler32
binary.BigEndian.PutUint32(b.tmp[0:], b.adler32.Sum32())
b.bytes.Write(b.tmp[0:4])
}
// A bitWriter is a write buffer for bit-oriented data like deflate.
type bitWriter struct {
bytes bytes.Buffer
bit uint32
nbit uint
tmp [4]byte
adler32 adigest
}
func (b *bitWriter) writeBits(bit uint32, nbit uint, rev bool) {
// reverse, for huffman codes
if rev {
br := uint32(0)
for i := uint(0); i < nbit; i++ {
br |= ((bit >> i) & 1) << (nbit - 1 - i)
}
bit = br
}
b.bit |= bit << b.nbit
b.nbit += nbit
for b.nbit >= 8 {
b.bytes.WriteByte(byte(b.bit))
b.bit >>= 8
b.nbit -= 8
}
}
func (b *bitWriter) flushBits() {
if b.nbit > 0 {
b.bytes.WriteByte(byte(b.bit))
b.nbit = 0
b.bit = 0
}
}
func (b *bitWriter) hcode(v int) {
/*
Lit Value Bits Codes
--------- ---- -----
0 - 143 8 00110000 through
10111111
144 - 255 9 110010000 through
111111111
256 - 279 7 0000000 through
0010111
280 - 287 8 11000000 through
11000111
*/
switch {
case v <= 143:
b.writeBits(uint32(v)+0x30, 8, true)
case v <= 255:
b.writeBits(uint32(v-144)+0x190, 9, true)
case v <= 279:
b.writeBits(uint32(v-256)+0, 7, true)
case v <= 287:
b.writeBits(uint32(v-280)+0xc0, 8, true)
default:
panic("invalid hcode")
}
}
func (b *bitWriter) byte(x byte) {
b.hcode(int(x))
}
func (b *bitWriter) codex(c int, val int, nx uint) {
b.hcode(c + val>>nx)
b.writeBits(uint32(val)&(1<<nx-1), nx, false)
}
func (b *bitWriter) repeat(n, d int) {
for ; n >= 258+3; n -= 258 {
b.repeat1(258, d)
}
if n > 258 {
// 258 < n < 258+3
b.repeat1(10, d)
b.repeat1(n-10, d)
return
}
if n < 3 {
panic("invalid flate repeat")
}
b.repeat1(n, d)
}
func (b *bitWriter) repeat1(n, d int) {
/*
Extra Extra Extra
Code Bits Length(s) Code Bits Lengths Code Bits Length(s)
---- ---- ------ ---- ---- ------- ---- ---- -------
257 0 3 267 1 15,16 277 4 67-82
258 0 4 268 1 17,18 278 4 83-98
259 0 5 269 2 19-22 279 4 99-114
260 0 6 270 2 23-26 280 4 115-130
261 0 7 271 2 27-30 281 5 131-162
262 0 8 272 2 31-34 282 5 163-194
263 0 9 273 3 35-42 283 5 195-226
264 0 10 274 3 43-50 284 5 227-257
265 1 11,12 275 3 51-58 285 0 258
266 1 13,14 276 3 59-66
*/
switch {
case n <= 10:
b.codex(257, n-3, 0)
case n <= 18:
b.codex(265, n-11, 1)
case n <= 34:
b.codex(269, n-19, 2)
case n <= 66:
b.codex(273, n-35, 3)
case n <= 130:
b.codex(277, n-67, 4)
case n <= 257:
b.codex(281, n-131, 5)
case n == 258:
b.hcode(285)
default:
panic("invalid repeat length")
}
/*
Extra Extra Extra
Code Bits Dist Code Bits Dist Code Bits Distance
---- ---- ---- ---- ---- ------ ---- ---- --------
0 0 1 10 4 33-48 20 9 1025-1536
1 0 2 11 4 49-64 21 9 1537-2048
2 0 3 12 5 65-96 22 10 2049-3072
3 0 4 13 5 97-128 23 10 3073-4096
4 1 5,6 14 6 129-192 24 11 4097-6144
5 1 7,8 15 6 193-256 25 11 6145-8192
6 2 9-12 16 7 257-384 26 12 8193-12288
7 2 13-16 17 7 385-512 27 12 12289-16384
8 3 17-24 18 8 513-768 28 13 16385-24576
9 3 25-32 19 8 769-1024 29 13 24577-32768
*/
if d <= 4 {
b.writeBits(uint32(d-1), 5, true)
} else if d <= 32768 {
nbit := uint(16)
for d <= 1<<(nbit-1) {
nbit--
}
v := uint32(d - 1)
v &^= 1 << (nbit - 1) // top bit is implicit
code := uint32(2*nbit - 2) // second bit is low bit of code
code |= v >> (nbit - 2)
v &^= 1 << (nbit - 2)
b.writeBits(code, 5, true)
// rest of bits follow
b.writeBits(uint32(v), nbit-2, false)
} else {
panic("invalid repeat distance")
}
}
func (b *bitWriter) run(v byte, n int) {
if n == 0 {
return
}
b.byte(v)
if n-1 < 3 {
for i := 0; i < n-1; i++ {
b.byte(v)
}
} else {
b.repeat(n-1, 1)
}
}
type adigest struct {
a, b uint32
}
func (d *adigest) Reset() { d.a, d.b = 1, 0 }
const amod = 65521
func aupdate(a, b uint32, pi byte, n int) (aa, bb uint32) {
// TODO(rsc): 6g doesn't do magic multiplies for b %= amod,
// only for b = b%amod.
// invariant: a, b < amod
if pi == 0 {
b += uint32(n%amod) * a
b = b % amod
return a, b
}
// n times:
// a += pi
// b += a
// is same as
// b += n*a + n*(n+1)/2*pi
// a += n*pi
m := uint32(n)
b += (m % amod) * a
b = b % amod
b += (m * (m + 1) / 2) % amod * uint32(pi)
b = b % amod
a += (m % amod) * uint32(pi)
a = a % amod
return a, b
}
func afinish(a, b uint32) uint32 {
return b<<16 | a
}
func (d *adigest) WriteN(p []byte, n int) {
for i := 0; i < n; i++ {
for _, pi := range p {
d.a, d.b = aupdate(d.a, d.b, pi, 1)
}
}
}
func (d *adigest) WriteNByte(pi byte, n int) {
d.a, d.b = aupdate(d.a, d.b, pi, n)
}
func (d *adigest) Sum32() uint32 { return afinish(d.a, d.b) }

109
Godeps/_workspace/src/github.com/vitrun/qart/qr/qr.go generated vendored Normal file
View File

@@ -0,0 +1,109 @@
package qr
import (
"errors"
"image"
"image/color"
"github.com/vitrun/qart/coding"
)
// A Level denotes a QR error correction level.
// From least to most tolerant of errors, they are L, M, Q, H.
type Level int
const (
L Level = iota // 20% redundant
M // 38% redundant
Q // 55% redundant
H // 65% redundant
)
// Encode returns an encoding of text at the given error correction level.
func Encode(text string, level Level) (*Code, error) {
// Pick data encoding, smallest first.
// We could split the string and use different encodings
// but that seems like overkill for now.
var enc coding.Encoding
switch {
case coding.Num(text).Check() == nil:
enc = coding.Num(text)
case coding.Alpha(text).Check() == nil:
enc = coding.Alpha(text)
default:
enc = coding.String(text)
}
// Pick size.
l := coding.Level(level)
var v coding.Version
for v = coding.MinVersion; ; v++ {
if v > coding.MaxVersion {
return nil, errors.New("text too long to encode as QR")
}
if enc.Bits(v) <= v.DataBytes(l)*8 {
break
}
}
// Build and execute plan.
p, err := coding.NewPlan(v, l, 0)
if err != nil {
return nil, err
}
cc, err := p.Encode(enc)
if err != nil {
return nil, err
}
// TODO: Pick appropriate mask.
return &Code{cc.Bitmap, cc.Size, cc.Stride, 8}, nil
}
// A Code is a square pixel grid.
// It implements image.Image and direct PNG encoding.
type Code struct {
Bitmap []byte // 1 is black, 0 is white
Size int // number of pixels on a side
Stride int // number of bytes per row
Scale int // number of image pixels per QR pixel
}
// Black returns true if the pixel at (x,y) is black.
func (c *Code) Black(x, y int) bool {
return 0 <= x && x < c.Size && 0 <= y && y < c.Size &&
c.Bitmap[y*c.Stride+x/8]&(1<<uint(7-x&7)) != 0
}
// Image returns an Image displaying the code.
func (c *Code) Image() image.Image {
return &codeImage{c}
}
// codeImage implements image.Image
type codeImage struct {
*Code
}
var (
whiteColor color.Color = color.Gray{0xFF}
blackColor color.Color = color.Gray{0x00}
)
func (c *codeImage) Bounds() image.Rectangle {
d := (c.Size + 8) * c.Scale
return image.Rect(0, 0, d, d)
}
func (c *codeImage) At(x, y int) color.Color {
if c.Black(x, y) {
return blackColor
}
return whiteColor
}
func (c *codeImage) ColorModel() color.Model {
return color.GrayModel
}

View File

@@ -0,0 +1,151 @@
package qr
import (
"image"
"image/color"
)
// average convert the sums to averages and returns the result.
func average(sum []uint64, w, h int, n uint64) *image.RGBA {
ret := image.NewRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
index := 4 * (y*w + x)
pix := ret.Pix[y*ret.Stride+x*4:]
pix[0] = uint8(sum[index+0] / n)
pix[1] = uint8(sum[index+1] / n)
pix[2] = uint8(sum[index+2] / n)
pix[3] = uint8(sum[index+3] / n)
}
}
return ret
}
// ResizeRGBA returns a scaled copy of the RGBA image slice r of m.
// The returned image has width w and height h.
func ResizeRGBA(m *image.RGBA, r image.Rectangle, w, h int) *image.RGBA {
ww, hh := uint64(w), uint64(h)
dx, dy := uint64(r.Dx()), uint64(r.Dy())
// See comment in Resize.
n, sum := dx*dy, make([]uint64, 4*w*h)
for y := r.Min.Y; y < r.Max.Y; y++ {
pix := m.Pix[(y-r.Min.Y)*m.Stride:]
for x := r.Min.X; x < r.Max.X; x++ {
// Get the source pixel.
p := pix[(x-r.Min.X)*4:]
r64 := uint64(p[0])
g64 := uint64(p[1])
b64 := uint64(p[2])
a64 := uint64(p[3])
// Spread the source pixel over 1 or more destination rows.
py := uint64(y) * hh
for remy := hh; remy > 0; {
qy := dy - (py % dy)
if qy > remy {
qy = remy
}
// Spread the source pixel over 1 or more destination columns.
px := uint64(x) * ww
index := 4 * ((py/dy)*ww + (px / dx))
for remx := ww; remx > 0; {
qx := dx - (px % dx)
if qx > remx {
qx = remx
}
qxy := qx * qy
sum[index+0] += r64 * qxy
sum[index+1] += g64 * qxy
sum[index+2] += b64 * qxy
sum[index+3] += a64 * qxy
index += 4
px += qx
remx -= qx
}
py += qy
remy -= qy
}
}
}
return average(sum, w, h, (uint64)(n))
}
// ResizeNRGBA returns a scaled copy of the RGBA image slice r of m.
// The returned image has width w and height h.
func ResizeNRGBA(m *image.NRGBA, r image.Rectangle, w, h int) *image.RGBA {
ww, hh := uint64(w), uint64(h)
dx, dy := uint64(r.Dx()), uint64(r.Dy())
// See comment in Resize.
n, sum := dx*dy, make([]uint64, 4*w*h)
for y := r.Min.Y; y < r.Max.Y; y++ {
pix := m.Pix[(y-r.Min.Y)*m.Stride:]
for x := r.Min.X; x < r.Max.X; x++ {
// Get the source pixel.
p := pix[(x-r.Min.X)*4:]
r64 := uint64(p[0])
g64 := uint64(p[1])
b64 := uint64(p[2])
a64 := uint64(p[3])
r64 = (r64 * a64) / 255
g64 = (g64 * a64) / 255
b64 = (b64 * a64) / 255
// Spread the source pixel over 1 or more destination rows.
py := uint64(y) * hh
for remy := hh; remy > 0; {
qy := dy - (py % dy)
if qy > remy {
qy = remy
}
// Spread the source pixel over 1 or more destination columns.
px := uint64(x) * ww
index := 4 * ((py/dy)*ww + (px / dx))
for remx := ww; remx > 0; {
qx := dx - (px % dx)
if qx > remx {
qx = remx
}
qxy := qx * qy
sum[index+0] += r64 * qxy
sum[index+1] += g64 * qxy
sum[index+2] += b64 * qxy
sum[index+3] += a64 * qxy
index += 4
px += qx
remx -= qx
}
py += qy
remy -= qy
}
}
}
return average(sum, w, h, (uint64)(n))
}
// Resample returns a resampled copy of the image slice r of m.
// The returned image has width w and height h.
func Resample(m image.Image, r image.Rectangle, w, h int) *image.RGBA {
if w < 0 || h < 0 {
return nil
}
if w == 0 || h == 0 || r.Dx() <= 0 || r.Dy() <= 0 {
return image.NewRGBA(image.Rect(0, 0, w, h))
}
curw, curh := r.Dx(), r.Dy()
img := image.NewRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
// Get a source pixel.
subx := x * curw / w
suby := y * curh / h
r32, g32, b32, a32 := m.At(subx, suby).RGBA()
r := uint8(r32 >> 8)
g := uint8(g32 >> 8)
b := uint8(b32 >> 8)
a := uint8(a32 >> 8)
img.SetRGBA(x, y, color.RGBA{r, g, b, a})
}
}
return img
}

View File

@@ -1,4 +1,4 @@
Copyright (C) 2013 Jakob Borg
Copyright (C) 2014 Jakob Borg and other contributors (see the CONTRIBUTORS file).
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in

179
README.md
View File

@@ -1,173 +1,48 @@
syncthing
=========
This is `syncthing`, an open BitTorrent Sync alternative. It is
currently far from ready for mass consumption, but it is a usable proof
of concept and tech demo. The following are the project goals:
This is the `syncthing` project. The following are the project goals:
1. Define an open, secure, language neutral protocol usable for
efficient synchronization of a file repository between an arbitrary
number of nodes. This is the [Block Exchange
Protocol](https://github.com/calmh/syncthing/blob/master/protocol/PROTOCOL.md)
(BEP).
1. Define a protocol for synchronization of a file repository between a
number of collaborating nodes. The protocol should be well defined,
unambiguous, easily understood, free to use, efficient, secure and
language neutral. This is the [Block Exchange
Protocol](https://github.com/calmh/syncthing/blob/master/protocol/PROTOCOL.md).
2. Provide the reference implementation to demonstrate the usability of
said protocol. This is the `syncthing` utility.
said protocol. This is the `syncthing` utility. It is the hope that
alternative, compatible implementations of the protocol will come to
exist.
The two are evolving together; the protocol is not to be considered
stable until syncthing 1.0 is released, at which point it is locked down
for incompatible changes.
Syncthing does not use the BitTorrent protocol. The reasons for this are
1) we don't know if BitTorrent Sync does either, so there's nothing to
be compatible with, 2) BitTorrent includes a lot of functionality for
making sure large swarms of selfish agents behave and somehow work
towards a common goal. Here we have a much smaller swarm of cooperative
agents and a simpler approach will suffice.
Getting Started
---------------
Features
--------
Take a look at the [getting started guide](http://discourse.syncthing.net/t/getting-started/46).
The following features are _currently implemented and working_:
Signed Releases
---------------
* The formation of a cluster of nodes, certificate authenticated and
communicating over TLS over TCP.
As of v0.7.0 and onwards, git tags and release binaries are GPG signed with
the key BCE524C7 (http://nym.se/gpg.txt). The signature is included in the
normal release bundle as `syncthing.asc` or `syncthing.exe.asc`.
* Synchronization of a single directory among the cluster nodes.
Documentation
=============
* Change detection by periodic scanning of the local repository.
* Static configuration of cluster nodes.
* Automatic discovery of cluster nodes. See [discover.go][discover.go]
for the protocol specification. Discovery on the LAN is performed by
broadcasts, Internet wide discovery is performed with the assistance
of a global server.
* Handling of deleted files. Deletes can be propagated or ignored per
client.
* Synchronizing multiple unrelated directory trees by following
symlinks directly below the repository level.
The following features are _not yet implemented but planned_:
* Change detection by listening to file system notifications instead of
periodic scanning.
* HTTP GUI.
The following features are _not implemented but may be implemented_ in
the future:
* Syncing multiple directories from the same syncthing instance.
* Automatic NAT handling via UPNP.
* Conflict resolution. Currently whichever file has the newest
modification time "wins". The correct behavior in the face of
conflicts is open for discussion.
[discover.go]: (https://github.com/calmh/syncthing/blob/master/discover/discover.go
Security
--------
Security is one of the primary project goals. This means that it should
not be possible for an attacker to join a cluster uninvited, and it
should not be possible to extract private information from intercepted
traffic. Currently this is implemented as follows.
All traffic is protected by TLS. To prevent uninvited nodes from joining
a cluster, the certificate fingerprint of each node is compared to a
preset list of acceptable nodes at connection establishment. The
fingerprint is computed as the SHA-1 hash of the certificate and
displayed in BASE32 encoding to form a compact yet convenient string.
Currently SHA-1 is deemed secure against preimage attacks.
Installing
==========
Download the appropriate precompiled binary from the
[releases](https://github.com/calmh/syncthing/releases) page. Untar and
put the `syncthing` binary somewhere convenient in your `$PATH`.
If you are a developer and have Go 1.2 installed you can also install
the latest version from source:
`go get github.com/calmh/syncthing`
Usage
=====
Check out the options:
```
$ syncthing --help
Usage:
syncthing [options]
...
```
Run syncthing to let it create it's config directory and certificate:
```
$ syncthing
11:34:13 main.go:85: INFO: Version v0.1-40-gbb0fd87
11:34:13 tls.go:61: OK: wrote cert.pem
11:34:13 tls.go:67: OK: wrote key.pem
11:34:13 main.go:66: INFO: My ID: NCTBZAAHXR6ZZP3D7SL3DLYFFQERMW4Q
11:34:13 main.go:90: FATAL: No config file
```
Take note of the "My ID: ..." line. Perform the same operation on
another computer to create another node. Take note of that ID as well,
and create a config file `~/.syncthing/syncthing.ini` looking something
like this:
```
[repository]
dir = /Users/jb/Synced
[nodes]
NCTBZAAHXR6ZZP3D7SL3DLYFFQERMW4Q = 172.16.32.1:22000 192.23.34.56:22000
CUGAE43Y5N64CRJU26YFH6MTWPSBLSUL = dynamic
```
This assumes that the first node is reachable on either of the two
addresses listed (perhaps one internal and one port-forwarded external)
and that the other node is not normally reachable from the outside. Save
this config file, identically, to both nodes.
If the nodes are running on the same network, or reachable on port 22000
from the outside world, you can set all addresses to "dynamic" and they
will find each other using automatic discovery. (This discovery,
including port numbers, can be tweaked or disabled using command line
options.)
Start syncthing on both nodes. For the cautious, one side can be set to
be read only.
```
$ syncthing --ro
13:30:55 main.go:85: INFO: Version v0.1-40-gbb0fd87
13:30:55 main.go:102: INFO: My ID: NCTBZAAHXR6ZZP3D7SL3DLYFFQERMW4Q
13:30:55 main.go:149: INFO: Initial repository scan in progress
13:30:59 main.go:153: INFO: Listening for incoming connections
13:30:59 main.go:157: INFO: Attempting to connect to other nodes
13:30:59 main.go:247: INFO: Starting local discovery
13:30:59 main.go:165: OK: Ready to synchronize
13:31:04 discover.go:113: INFO: Discovered node CUGAE43Y5N64CRJU26YFH6MTWPSBLSUL at 172.16.32.24:22000
13:31:14 main.go:296: OK: Connected to node CUGAE43Y5N64CRJU26YFH6MTWPSBLSUL
13:31:19 main.go:345: INFO: Transferred 139 KiB in (14 KiB/s), 139 KiB out (14 KiB/s)
...
```
You should see the synchronization start and then finish a short while
later. Add nodes to taste.
The [syncthing
documentation](http://discourse.syncthing.net/category/documentation) is
on the discourse site.
License
=======
MIT
All documentation and protocol specifications are licensed
under the [Creative Commons Attribution 4.0 International
License](http://creativecommons.org/licenses/by/4.0/).
All code is licensed under the [MIT
License](https://github.com/calmh/syncthing/blob/master/LICENSE).

View File

@@ -0,0 +1,14 @@
# editorconfig.org
root = true
[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
[*.py]
indent_size = 4

8
assets/bootstrap-3.1.1/.gitattributes vendored Normal file
View File

@@ -0,0 +1,8 @@
# Enforce Unix newlines
*.css text eol=lf
*.html text eol=lf
*.js text eol=lf
*.json text eol=lf
*.less text eol=lf
*.md text eol=lf
*.yml text eol=lf

42
assets/bootstrap-3.1.1/.gitignore vendored Normal file
View File

@@ -0,0 +1,42 @@
# Ignore docs files
_gh_pages
_site
.ruby-version
# Numerous always-ignore extensions
*.diff
*.err
*.orig
*.log
*.rej
*.swo
*.swp
*.zip
*.vi
*~
# OS or Editor folders
.DS_Store
._*
Thumbs.db
.cache
.project
.settings
.tmproj
*.esproj
nbproject
*.sublime-project
*.sublime-workspace
.idea
# Komodo
*.komodoproject
.komodotools
# grunt-html-validation
validation-status.json
validation-report.json
# Folders to ignore
node_modules
bower_components

View File

@@ -0,0 +1,28 @@
language: node_js
node_js:
- "0.10"
before_install:
- time sudo pip install --use-mirrors -r test-infra/requirements.txt
- rvm use 1.9.3 --fuzzy
- if [ "$TWBS_TEST" = validate-html ]; then echo "ruby=$(basename $(rvm gemdir)) jekyll=$JEKYLL_VERSION" > pseudo_Gemfile.lock; fi
install:
- time npm install -g grunt-cli
- time ./test-infra/s3_cache.py download 'npm packages' test-infra/npm-shrinkwrap.canonical.json ./node_modules || time ./test-infra/uncached-npm-install.sh
- if [ "$TWBS_TEST" = validate-html ]; then time ./test-infra/s3_cache.py download rubygems pseudo_Gemfile.lock $(rvm gemdir) || gem install -N jekyll -v $JEKYLL_VERSION; fi
after_script:
- if [ "$TWBS_TEST" = core ]; then time ./test-infra/s3_cache.py upload 'npm packages' test-infra/npm-shrinkwrap.canonical.json ./node_modules; fi
- if [ "$TWBS_TEST" = validate-html ]; then time ./test-infra/s3_cache.py upload rubygems pseudo_Gemfile.lock $(rvm gemdir); fi
env:
global:
- JEKYLL_VERSION: 1.4.1
- SAUCE_USERNAME: bootstrap
- secure: "pJkBwnuae9dKU5tEcCqccfS1QQw7/meEcfz63fM7ba7QJNjoA6BaXj08L5Z3Vb5vBmVPwBawxo5Hp0jC0r/Z/O0hGnAmz/Cz09L+cy7dSAZ9x4hvZePSja/UAusaB5ogMoO8l2b773MzgQeSmrLbExr9BWLeqEfjC2hFgdgHLaQ="
- secure: "gqjqISbxBJK6byFbsmr1AyP1qoWH+rap06A2gI7v72+Tn2PU2nYkIMUkCvhZw6K889jv+LhQ/ybcBxDOXHpNCExCnSgB4dcnmYp+9oeNZb37jSP0rQ+Ib4OTLjzc3/FawE/fUq5kukZTC7porzc/k0qJNLAZRx3YLALmK1GIdUY="
- secure: "Gghh/e3Gsbj1+4RR9Lh2aR/xJl35HWiHqlPIeSUqE9D7uDCVTAwNce/dGL3Ew7uJPfJ6Pgr70wD3zgu3stw0Zmzayax0hiDtGwcQCxVIER08wqGANK9C2Q7PYJkNTNtiTo6ehKWbdV4Z+/U+TEYyQfpQTDbAFYk/vVpsdjp0Lmc="
- secure: "RTbRdx4G/2OTLfrZtP1VbRljxEmd6A1F3GqXboeQTldsnAlwpsES65es5CE3ub/rmixLApOY9ot7OPmNixFgC2Y8xOsV7lNCC62QVpmqQEDyGFFQKb3yO6/dmwQxdsCqGfzf9Np6Wh5V22QFvr50ZLKLd7Uhd9oXMDIk/z1MJ3o="
matrix:
- TWBS_TEST=core
- TWBS_TEST=validate-html
- TWBS_TEST=sauce-js-unit
matrix:
fast_finish: true

View File

@@ -0,0 +1 @@
getbootstrap.com

View File

@@ -0,0 +1,196 @@
# Contributing to Bootstrap
Looking to contribute something to Bootstrap? **Here's how you can help.**
Please take a moment to review this document in order to make the contribution
process easy and effective for everyone involved.
Following these guidelines helps to communicate that you respect the time of
the developers managing and developing this open source project. In return,
they should reciprocate that respect in addressing your issue or assessing
patches and features.
## Using the issue tracker
The [issue tracker](https://github.com/twbs/bootstrap/issues) is
the preferred channel for [bug reports](#bug-reports), [features requests](#feature-requests)
and [submitting pull requests](#pull-requests), but please respect the following
restrictions:
* Please **do not** use the issue tracker for personal support requests. Stack
Overflow ([`twitter-bootstrap-3`](http://stackoverflow.com/questions/tagged/twitter-bootstrap-3) tag) or [IRC](https://github.com/twbs/bootstrap/blob/master/README.md#community) are better places to get help.
* Please **do not** derail or troll issues. Keep the discussion on topic and
respect the opinions of others.
* Please **do not** open issues or pull requests regarding the code in
[`Normalize`](https://github.com/necolas/normalize.css) (open them in
their respective repositories).
## Bug reports
A bug is a _demonstrable problem_ that is caused by the code in the repository.
Good bug reports are extremely helpful, so thanks!
Guidelines for bug reports:
1. **Use the GitHub issue search** &mdash; check if the issue has already been
reported.
2. **Check if the issue has been fixed** &mdash; try to reproduce it using the
latest `master` or development branch in the repository.
3. **Isolate the problem** &mdash; ideally create a [reduced test
case](http://css-tricks.com/6263-reduced-test-cases/) and a live example.
[This JS Bin](http://jsbin.com/EBAwOkOK/1) is a helpful template.
A good bug report shouldn't leave others needing to chase you up for more
information. Please try to be as detailed as possible in your report. What is
your environment? What steps will reproduce the issue? What browser(s) and OS
experience the problem? Do other browsers show the bug differently? What
would you expect to be the outcome? All these details will help people to fix
any potential bugs.
Example:
> Short and descriptive example bug report title
>
> A summary of the issue and the browser/OS environment in which it occurs. If
> suitable, include the steps required to reproduce the bug.
>
> 1. This is the first step
> 2. This is the second step
> 3. Further steps, etc.
>
> `<url>` - a link to the reduced test case
>
> Any other information you want to share that is relevant to the issue being
> reported. This might include the lines of code that you have identified as
> causing the bug, and potential solutions (and your opinions on their
> merits).
## Feature requests
Feature requests are welcome. But take a moment to find out whether your idea
fits with the scope and aims of the project. It's up to *you* to make a strong
case to convince the project's developers of the merits of this feature. Please
provide as much detail and context as possible.
## Pull requests
Good pull requests—patches, improvements, new features—are a fantastic
help. They should remain focused in scope and avoid containing unrelated
commits.
**Please ask first** before embarking on any significant pull request (e.g.
implementing features, refactoring code, porting to a different language),
otherwise you risk spending a lot of time working on something that the
project's developers might not want to merge into the project.
Please adhere to the [coding guidelines](#code-guidelines) used throughout the
project (indentation, accurate comments, etc.) and any other requirements
(such as test coverage).
Adhering to the following process is the best way to get your work
included in the project:
1. [Fork](http://help.github.com/fork-a-repo/) the project, clone your fork,
and configure the remotes:
```bash
# Clone your fork of the repo into the current directory
git clone https://github.com/<your-username>/bootstrap.git
# Navigate to the newly cloned directory
cd bootstrap
# Assign the original repo to a remote called "upstream"
git remote add upstream https://github.com/twbs/bootstrap.git
```
2. If you cloned a while ago, get the latest changes from upstream:
```bash
git checkout master
git pull upstream master
```
3. Create a new topic branch (off the main project development branch) to
contain your feature, change, or fix:
```bash
git checkout -b <topic-branch-name>
```
4. Commit your changes in logical chunks. Please adhere to these [git commit
message guidelines](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html)
or your code is unlikely be merged into the main project. Use Git's
[interactive rebase](https://help.github.com/articles/interactive-rebase)
feature to tidy up your commits before making them public.
5. Locally merge (or rebase) the upstream development branch into your topic branch:
```bash
git pull [--rebase] upstream master
```
6. Push your topic branch up to your fork:
```bash
git push origin <topic-branch-name>
```
7. [Open a Pull Request](https://help.github.com/articles/using-pull-requests/)
with a clear title and description against the `master` branch.
**IMPORTANT**: By submitting a patch, you agree to allow the project owners to
license your work under the terms of the [MIT License](LICENSE.md).
## Code guidelines
### HTML
- Two spaces for indentation, never tabs.
- Double quotes only, never single quotes.
- Always use proper indentation.
- Use tags and elements appropriate for an HTML5 doctype (e.g., self-closing tags).
- Use CDNs and HTTPS for third-party JS when possible. We don't use protocol-relative URLs in this case because they break when viewing the page locally via `file://`.
- Use [WAI-ARIA](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) attributes in documentation examples to promote accessibility.
### CSS
- CSS changes must be done in `.less` files first, never just in the compiled `.css` files.
- Adhere to the [CSS property order](http://markdotto.com/2011/11/29/css-property-order/).
- Multiple-line approach (one property and value per line).
- Always a space after a property's colon (e.g., `display: block;` and not `display:block;`).
- End all lines with a semi-colon.
- For multiple, comma-separated selectors, place each selector on its own line.
- Attribute selectors, like `input[type="text"]` should always wrap the attribute's value in double quotes, for consistency and safety (see this [blog post on unquoted attribute values](http://mathiasbynens.be/notes/unquoted-attribute-values) that can lead to XSS attacks).
- Attribute selectors should only be used where absolutely necessary (e.g., form controls) and should be avoided on custom components for performance and explicitness.
- Series of classes for a component should include a base class (e.g., `.component`) and use the base class as a prefix for modifier and sub-components (e.g., `.component-lg`).
- Avoid inheritance and over nesting—use single, explicit classes whenever possible.
- When feasible, default color palettes should comply with [WCAG color contrast guidelines](http://www.w3.org/TR/WCAG20/#visual-audio-contrast).
- Except in rare cases, don't remove default `:focus` styles (via e.g. `outline: none;`) without providing alternative styles. See [this A11Y Project post](http://a11yproject.com/posts/never-remove-css-outlines/) for more details.
### JS
- No semicolons (in client-side JS)
- 2 spaces (no tabs)
- strict mode
- "Attractive"
### Checking coding style
Run `grunt test` before committing to ensure your changes follow our coding standards.
## License
By contributing your code, you agree to license your contribution under the [MIT license](https://github.com/twbs/bootstrap/blob/master/LICENSE).
Prior to v3.1.0, Bootstrap was released under the Apache License v2.0.

421
assets/bootstrap-3.1.1/Gruntfile.js vendored Normal file
View File

@@ -0,0 +1,421 @@
/*!
* Bootstrap's Gruntfile
* http://getbootstrap.com
* Copyright 2013-2014 Twitter, Inc.
* Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
*/
module.exports = function (grunt) {
'use strict';
// Force use of Unix newlines
grunt.util.linefeed = '\n';
RegExp.quote = function (string) {
return string.replace(/[-\\^$*+?.()|[\]{}]/g, '\\$&');
};
var fs = require('fs');
var path = require('path');
var generateGlyphiconsData = require('./grunt/bs-glyphicons-data-generator.js');
var BsLessdocParser = require('./grunt/bs-lessdoc-parser.js');
var generateRawFilesJs = require('./grunt/bs-raw-files-generator.js');
var updateShrinkwrap = require('./grunt/shrinkwrap.js');
// Project configuration.
grunt.initConfig({
// Metadata.
pkg: grunt.file.readJSON('package.json'),
banner: '/*!\n' +
' * Bootstrap v<%= pkg.version %> (<%= pkg.homepage %>)\n' +
' * Copyright 2011-<%= grunt.template.today("yyyy") %> <%= pkg.author %>\n' +
' * Licensed under <%= pkg.license.type %> (<%= pkg.license.url %>)\n' +
' */\n',
jqueryCheck: 'if (typeof jQuery === \'undefined\') { throw new Error(\'Bootstrap\\\'s JavaScript requires jQuery\') }\n\n',
// Task configuration.
clean: {
dist: ['dist', 'docs/dist']
},
jshint: {
options: {
jshintrc: 'js/.jshintrc'
},
grunt: {
options: {
jshintrc: 'grunt/.jshintrc'
},
src: ['Gruntfile.js', 'grunt/*.js']
},
src: {
src: 'js/*.js'
},
test: {
src: 'js/tests/unit/*.js'
},
assets: {
src: ['docs/assets/js/application.js', 'docs/assets/js/customizer.js']
}
},
jscs: {
options: {
config: 'js/.jscs.json',
},
grunt: {
src: ['Gruntfile.js', 'grunt/*.js']
},
src: {
src: 'js/*.js'
},
test: {
src: 'js/tests/unit/*.js'
},
assets: {
src: ['docs/assets/js/application.js', 'docs/assets/js/customizer.js']
}
},
csslint: {
options: {
csslintrc: 'less/.csslintrc'
},
src: [
'dist/css/bootstrap.css',
'dist/css/bootstrap-theme.css',
'docs/assets/css/docs.css',
'docs/examples/**/*.css'
]
},
concat: {
options: {
banner: '<%= banner %>\n<%= jqueryCheck %>',
stripBanners: false
},
bootstrap: {
src: [
'js/transition.js',
'js/alert.js',
'js/button.js',
'js/carousel.js',
'js/collapse.js',
'js/dropdown.js',
'js/modal.js',
'js/tooltip.js',
'js/popover.js',
'js/scrollspy.js',
'js/tab.js',
'js/affix.js'
],
dest: 'dist/js/<%= pkg.name %>.js'
}
},
uglify: {
options: {
report: 'min'
},
bootstrap: {
options: {
banner: '<%= banner %>'
},
src: '<%= concat.bootstrap.dest %>',
dest: 'dist/js/<%= pkg.name %>.min.js'
},
customize: {
options: {
preserveComments: 'some'
},
src: [
'docs/assets/js/vendor/less.min.js',
'docs/assets/js/vendor/jszip.min.js',
'docs/assets/js/vendor/uglify.min.js',
'docs/assets/js/vendor/blob.js',
'docs/assets/js/vendor/filesaver.js',
'docs/assets/js/raw-files.min.js',
'docs/assets/js/customizer.js'
],
dest: 'docs/assets/js/customize.min.js'
},
docsJs: {
options: {
preserveComments: 'some'
},
src: [
'docs/assets/js/vendor/holder.js',
'docs/assets/js/application.js'
],
dest: 'docs/assets/js/docs.min.js'
}
},
less: {
compileCore: {
options: {
strictMath: true,
sourceMap: true,
outputSourceFiles: true,
sourceMapURL: '<%= pkg.name %>.css.map',
sourceMapFilename: 'dist/css/<%= pkg.name %>.css.map'
},
files: {
'dist/css/<%= pkg.name %>.css': 'less/bootstrap.less'
}
},
compileTheme: {
options: {
strictMath: true,
sourceMap: true,
outputSourceFiles: true,
sourceMapURL: '<%= pkg.name %>-theme.css.map',
sourceMapFilename: 'dist/css/<%= pkg.name %>-theme.css.map'
},
files: {
'dist/css/<%= pkg.name %>-theme.css': 'less/theme.less'
}
},
minify: {
options: {
cleancss: true,
report: 'min'
},
files: {
'dist/css/<%= pkg.name %>.min.css': 'dist/css/<%= pkg.name %>.css',
'dist/css/<%= pkg.name %>-theme.min.css': 'dist/css/<%= pkg.name %>-theme.css'
}
}
},
cssmin: {
compress: {
options: {
keepSpecialComments: '*',
noAdvanced: true, // turn advanced optimizations off until the issue is fixed in clean-css
report: 'min',
selectorsMergeMode: 'ie8'
},
src: [
'docs/assets/css/docs.css',
'docs/assets/css/pygments-manni.css'
],
dest: 'docs/assets/css/docs.min.css'
}
},
usebanner: {
dist: {
options: {
position: 'top',
banner: '<%= banner %>'
},
files: {
src: [
'dist/css/<%= pkg.name %>.css',
'dist/css/<%= pkg.name %>.min.css',
'dist/css/<%= pkg.name %>-theme.css',
'dist/css/<%= pkg.name %>-theme.min.css'
]
}
}
},
csscomb: {
options: {
config: 'less/.csscomb.json'
},
dist: {
files: {
'dist/css/<%= pkg.name %>.css': 'dist/css/<%= pkg.name %>.css',
'dist/css/<%= pkg.name %>-theme.css': 'dist/css/<%= pkg.name %>-theme.css'
}
},
examples: {
expand: true,
cwd: 'docs/examples/',
src: ['**/*.css'],
dest: 'docs/examples/'
}
},
copy: {
fonts: {
expand: true,
src: 'fonts/*',
dest: 'dist/'
},
docs: {
expand: true,
cwd: './dist',
src: [
'{css,js}/*.min.*',
'css/*.map',
'fonts/*'
],
dest: 'docs/dist'
}
},
qunit: {
options: {
inject: 'js/tests/unit/phantom.js'
},
files: 'js/tests/index.html'
},
connect: {
server: {
options: {
port: 3000,
base: '.'
}
}
},
jekyll: {
docs: {}
},
jade: {
compile: {
options: {
pretty: true,
data: function () {
var filePath = path.join(__dirname, 'less/variables.less');
var fileContent = fs.readFileSync(filePath, {encoding: 'utf8'});
var parser = new BsLessdocParser(fileContent);
return {sections: parser.parseFile()};
}
},
files: {
'docs/_includes/customizer-variables.html': 'docs/jade/customizer-variables.jade',
'docs/_includes/nav-customize.html': 'docs/jade/customizer-nav.jade'
}
}
},
validation: {
options: {
charset: 'utf-8',
doctype: 'HTML5',
failHard: true,
reset: true,
relaxerror: [
'Bad value X-UA-Compatible for attribute http-equiv on element meta.',
'Element img is missing required attribute src.'
]
},
files: {
src: '_gh_pages/**/*.html'
}
},
watch: {
src: {
files: '<%= jshint.src.src %>',
tasks: ['jshint:src', 'qunit']
},
test: {
files: '<%= jshint.test.src %>',
tasks: ['jshint:test', 'qunit']
},
less: {
files: 'less/*.less',
tasks: 'less'
}
},
sed: {
versionNumber: {
pattern: (function () {
var old = grunt.option('oldver');
return old ? RegExp.quote(old) : old;
})(),
replacement: grunt.option('newver'),
recursive: true
}
},
'saucelabs-qunit': {
all: {
options: {
build: process.env.TRAVIS_JOB_ID,
concurrency: 10,
urls: ['http://127.0.0.1:3000/js/tests/index.html'],
browsers: grunt.file.readYAML('test-infra/sauce_browsers.yml')
}
}
},
exec: {
npmUpdate: {
command: 'npm update'
},
npmShrinkWrap: {
command: 'npm shrinkwrap --dev'
}
}
});
// These plugins provide necessary tasks.
require('load-grunt-tasks')(grunt, {scope: 'devDependencies'});
// Docs HTML validation task
grunt.registerTask('validate-html', ['jekyll', 'validation']);
// Test task.
var testSubtasks = [];
// Skip core tests if running a different subset of the test suite
if (!process.env.TWBS_TEST || process.env.TWBS_TEST === 'core') {
testSubtasks = testSubtasks.concat(['dist-css', 'csslint', 'jshint', 'jscs', 'qunit', 'build-customizer-html']);
}
// Skip HTML validation if running a different subset of the test suite
if (!process.env.TWBS_TEST || process.env.TWBS_TEST === 'validate-html') {
testSubtasks.push('validate-html');
}
// Only run Sauce Labs tests if there's a Sauce access key
if (typeof process.env.SAUCE_ACCESS_KEY !== 'undefined' &&
// Skip Sauce if running a different subset of the test suite
(!process.env.TWBS_TEST || process.env.TWBS_TEST === 'sauce-js-unit')) {
testSubtasks.push('connect');
testSubtasks.push('saucelabs-qunit');
}
grunt.registerTask('test', testSubtasks);
// JS distribution task.
grunt.registerTask('dist-js', ['concat', 'uglify']);
// CSS distribution task.
grunt.registerTask('dist-css', ['less', 'cssmin', 'csscomb', 'usebanner']);
// Docs distribution task.
grunt.registerTask('dist-docs', 'copy:docs');
// Full distribution task.
grunt.registerTask('dist', ['clean', 'dist-css', 'copy:fonts', 'dist-js', 'dist-docs']);
// Default task.
grunt.registerTask('default', ['test', 'dist', 'build-glyphicons-data', 'build-customizer', 'update-shrinkwrap']);
// Version numbering task.
// grunt change-version-number --oldver=A.B.C --newver=X.Y.Z
// This can be overzealous, so its changes should always be manually reviewed!
grunt.registerTask('change-version-number', 'sed');
grunt.registerTask('build-glyphicons-data', generateGlyphiconsData);
// task for building customizer
grunt.registerTask('build-customizer', ['build-customizer-html', 'build-raw-files']);
grunt.registerTask('build-customizer-html', 'jade');
grunt.registerTask('build-raw-files', 'Add scripts/less files to customizer.', function () {
var banner = grunt.template.process('<%= banner %>');
generateRawFilesJs(banner);
});
// Task for updating the npm packages used by the Travis build.
grunt.registerTask('update-shrinkwrap', ['exec:npmUpdate', 'exec:npmShrinkWrap', '_update-shrinkwrap']);
grunt.registerTask('_update-shrinkwrap', function () { updateShrinkwrap.call(this, grunt); });
};

View File

@@ -0,0 +1,21 @@
The MIT License (MIT)
Copyright (c) 2011-2014 Twitter, Inc
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

View File

@@ -0,0 +1,173 @@
# [Bootstrap](http://getbootstrap.com) [![Bower version](https://badge.fury.io/bo/bootstrap.png)](http://badge.fury.io/bo/bootstrap) [![Build Status](https://secure.travis-ci.org/twbs/bootstrap.png)](http://travis-ci.org/twbs/bootstrap) [![devDependency Status](https://david-dm.org/twbs/bootstrap/dev-status.png?theme=shields.io)](https://david-dm.org/twbs/bootstrap#info=devDependencies)
[![Selenium Test Status](https://saucelabs.com/browser-matrix/bootstrap.svg)](https://saucelabs.com/u/bootstrap)
Bootstrap is a sleek, intuitive, and powerful front-end framework for faster and easier web development, created by [Mark Otto](http://twitter.com/mdo) and [Jacob Thornton](http://twitter.com/fat), and maintained by the [core team](https://github.com/twbs?tab=members) with the massive support and involvement of the community.
To get started, check out <http://getbootstrap.com>!
## Table of contents
- [Quick start](#quick-start)
- [Bugs and feature requests](#bugs-and-feature-requests)
- [Documentation](#documentation)
- [Compiling CSS and JavaScript](#compiling-css-and-javascript)
- [Contributing](#contributing)
- [Community](#community)
- [Versioning](#versioning)
- [Authors](#authors)
- [Copyright and license](#copyright-and-license)
## Quick start
Three quick start options are available:
- [Download the latest release](https://github.com/twbs/bootstrap/archive/v3.1.1.zip).
- Clone the repo: `git clone https://github.com/twbs/bootstrap.git`.
- Install with [Bower](http://bower.io): `bower install bootstrap`.
Read the [Getting Started page](http://getbootstrap.com/getting-started/) for information on the framework contents, templates and examples, and more.
### What's included
Within the download you'll find the following directories and files, logically grouping common assets and providing both compiled and minified variations. You'll see something like this:
```
bootstrap/
├── css/
│ ├── bootstrap.css
│ ├── bootstrap.min.css
│ ├── bootstrap-theme.css
│ └── bootstrap-theme.min.css
├── js/
│ ├── bootstrap.js
│ └── bootstrap.min.js
└── fonts/
├── glyphicons-halflings-regular.eot
├── glyphicons-halflings-regular.svg
├── glyphicons-halflings-regular.ttf
└── glyphicons-halflings-regular.woff
```
We provide compiled CSS and JS (`bootstrap.*`), as well as compiled and minified CSS and JS (`bootstrap.min.*`). Fonts from Glyphicons are included, as is the optional Bootstrap theme.
## Bugs and feature requests
Have a bug or a feature request? Please first read the [issue guidelines](https://github.com/twbs/bootstrap/blob/master/CONTRIBUTING.md#using-the-issue-tracker) and search for existing and closed issues. If your problem or idea is not addressed yet, [please open a new issue](https://github.com/twbs/bootstrap/issues/new).
## Documentation
Bootstrap's documentation, included in this repo in the root directory, is built with [Jekyll](http://jekyllrb.com) and publicly hosted on GitHub Pages at <http://getbootstrap.com>. The docs may also be run locally.
### Running documentation locally
1. If necessary, [install Jekyll](http://jekyllrb.com/docs/installation) (requires v1.x).
- **Windows users:** Read [this unofficial guide](https://github.com/juthilo/run-jekyll-on-windows/) to get Jekyll up and running without problems. We use Pygments for syntax highlighting, so make sure to read the sections on installing Python and Pygments.
2. From the root `/bootstrap` directory, run `jekyll serve` in the command line.
- **Windows users:** While we use Jekyll's `encoding` setting, you might still need to change the command prompt's character encoding ([code page](http://en.wikipedia.org/wiki/Windows_code_page)) to UTF-8 so Jekyll runs without errors. For Ruby 2.0.0, run `chcp 65001` first. For Ruby 1.9.3, you can alternatively do `SET LANG=en_EN.UTF-8`.
3. Open <http://localhost:9001> in your browser, and voilà.
Learn more about using Jekyll by reading its [documentation](http://jekyllrb.com/docs/home/).
### Documentation for previous releases
Documentation for v2.3.2 has been made available for the time being at <http://getbootstrap.com/2.3.2/> while folks transition to Bootstrap 3.
[Previous releases](https://github.com/twbs/bootstrap/releases) and their documentation are also available for download.
## Compiling CSS and JavaScript
Bootstrap uses [Grunt](http://gruntjs.com/) with convenient methods for working with the framework. It's how we compile our code, run tests, and more. To use it, install the required dependencies as directed and then run some Grunt commands.
### Install Grunt
From the command line:
1. Install `grunt-cli` globally with `npm install -g grunt-cli`.
2. Navigate to the root `/bootstrap` directory, then run `npm install`. npm will look at [package.json](https://github.com/twbs/bootstrap/blob/master/package.json) and automatically install the necessary local dependencies listed there.
When completed, you'll be able to run the various Grunt commands provided from the command line.
**Unfamiliar with `npm`? Don't have node installed?** That's a-okay. npm stands for [node packaged modules](http://npmjs.org/) and is a way to manage development dependencies through node.js. [Download and install node.js](http://nodejs.org/download/) before proceeding.
### Available Grunt commands
#### Build - `grunt`
Run `grunt` to run tests locally and compile the CSS and JavaScript into `/dist`. **Uses [Less](http://lesscss.org/) and [UglifyJS](http://lisperator.net/uglifyjs/).**
#### Only compile CSS and JavaScript - `grunt dist`
`grunt dist` creates the `/dist` directory with compiled files. **Uses [Less](http://lesscss.org/) and [UglifyJS](http://lisperator.net/uglifyjs/).**
#### Tests - `grunt test`
Runs [JSHint](http://jshint.com) and [QUnit](http://qunitjs.com/) tests headlessly in [PhantomJS](http://phantomjs.org/) (used for CI).
#### Watch - `grunt watch`
This is a convenience method for watching just Less files and automatically building them whenever you save.
### Troubleshooting dependencies
Should you encounter problems with installing dependencies or running Grunt commands, uninstall all previous dependency versions (global and local). Then, rerun `npm install`.
## Contributing
Please read through our [contributing guidelines](https://github.com/twbs/bootstrap/blob/master/CONTRIBUTING.md). Included are directions for opening issues, coding standards, and notes on development.
Moreover, if your pull request contains JavaScript patches or features, you must include relevant unit tests. All HTML and CSS should conform to the [Code Guide](http://github.com/mdo/code-guide), maintained by [Mark Otto](http://github.com/mdo).
Editor preferences are available in the [editor config](https://github.com/twbs/bootstrap/blob/master/.editorconfig) for easy use in common text editors. Read more and download plugins at <http://editorconfig.org>.
## Community
Keep track of development and community news.
- Follow [@twbootstrap on Twitter](http://twitter.com/twbootstrap).
- Read and subscribe to [The Official Bootstrap Blog](http://blog.getbootstrap.com).
- Chat with fellow Bootstrappers in IRC. On the `irc.freenode.net` server, in the `##twitter-bootstrap` channel.
- Implementation help may be found at Stack Overflow (tagged [`twitter-bootstrap-3`](http://stackoverflow.com/questions/tagged/twitter-bootstrap-3)).
## Versioning
For transparency into our release cycle and in striving to maintain backward compatibility, Bootstrap is maintained under the Semantic Versioning guidelines. Sometimes we screw up, but we'll adhere to these rules whenever possible.
Releases will be numbered with the following format:
`<major>.<minor>.<patch>`
And constructed with the following guidelines:
- Breaking backward compatibility **bumps the major** while resetting minor and patch
- New additions without breaking backward compatibility **bumps the minor** while resetting the patch
- Bug fixes and misc changes **bumps only the patch**
For more information on SemVer, please visit <http://semver.org/>.
## Authors
**Mark Otto**
- <http://twitter.com/mdo>
- <http://github.com/mdo>
**Jacob Thornton**
- <http://twitter.com/fat>
- <http://github.com/fat>
## Copyright and license
Code and documentation copyright 2011-2014 Twitter, Inc. Code released under [the MIT license](LICENSE). Docs released under [Creative Commons](docs/LICENSE).

View File

@@ -0,0 +1,37 @@
# Dependencies
markdown: rdiscount
pygments: true
# Permalinks
permalink: pretty
# Server
source: ./docs
destination: ./_gh_pages
host: 0.0.0.0
port: 9001
baseurl: /
url: http://localhost:9001
encoding: UTF-8
exclude:
- "jade"
- "vendor"
# Custom vars
current_version: 3.1.1
repo: https://github.com/twbs/bootstrap
sass_repo: https://github.com/twbs/bootstrap-sass
download:
source: https://github.com/twbs/bootstrap/archive/v3.1.1.zip
dist: https://github.com/twbs/bootstrap/releases/download/v3.1.1/bootstrap-3.1.1-dist.zip
sass: https://github.com/twbs/bootstrap-sass/archive/v3.1.1.tar.gz
blog: http://blog.getbootstrap.com
expo: http://expo.getbootstrap.com
cdn:
css: //netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css
css_theme: //netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap-theme.min.css
js: //netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js

View File

@@ -0,0 +1,24 @@
{
"name": "bootstrap",
"version": "3.1.1",
"main": [
"./dist/css/bootstrap.css",
"./dist/js/bootstrap.js",
"./dist/fonts/glyphicons-halflings-regular.eot",
"./dist/fonts/glyphicons-halflings-regular.svg",
"./dist/fonts/glyphicons-halflings-regular.ttf",
"./dist/fonts/glyphicons-halflings-regular.woff"
],
"ignore": [
"**/.*",
"_config.yml",
"CNAME",
"composer.json",
"CONTRIBUTING.md",
"docs",
"js/tests"
],
"dependencies": {
"jquery": ">= 1.9.0"
}
}

View File

@@ -0,0 +1,25 @@
{
"name": "twbs/bootstrap",
"description": "Sleek, intuitive, and powerful mobile first front-end framework for faster and easier web development.",
"keywords": ["bootstrap", "css"],
"homepage": "http://getbootstrap.com",
"authors": [
{
"name": "Mark Otto",
"email": "markdotto@gmail.com"
},
{
"name": "Jacob Thornton",
"email": "jacobthornton@gmail.com"
}
],
"support": {
"issues": "https://github.com/twbs/bootstrap/issues"
},
"license": "MIT",
"extra": {
"branch-alias": {
"dev-master": "3.0.x-dev"
}
}
}

View File

@@ -0,0 +1,347 @@
/*!
* Bootstrap v3.1.1 (http://getbootstrap.com)
* Copyright 2011-2014 Twitter, Inc.
* Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE)
*/
.btn-default,
.btn-primary,
.btn-success,
.btn-info,
.btn-warning,
.btn-danger {
text-shadow: 0 -1px 0 rgba(0, 0, 0, .2);
-webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, .15), 0 1px 1px rgba(0, 0, 0, .075);
box-shadow: inset 0 1px 0 rgba(255, 255, 255, .15), 0 1px 1px rgba(0, 0, 0, .075);
}
.btn-default:active,
.btn-primary:active,
.btn-success:active,
.btn-info:active,
.btn-warning:active,
.btn-danger:active,
.btn-default.active,
.btn-primary.active,
.btn-success.active,
.btn-info.active,
.btn-warning.active,
.btn-danger.active {
-webkit-box-shadow: inset 0 3px 5px rgba(0, 0, 0, .125);
box-shadow: inset 0 3px 5px rgba(0, 0, 0, .125);
}
.btn:active,
.btn.active {
background-image: none;
}
.btn-default {
text-shadow: 0 1px 0 #fff;
background-image: -webkit-linear-gradient(top, #fff 0%, #e0e0e0 100%);
background-image: linear-gradient(to bottom, #fff 0%, #e0e0e0 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffffffff', endColorstr='#ffe0e0e0', GradientType=0);
filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
background-repeat: repeat-x;
border-color: #dbdbdb;
border-color: #ccc;
}
.btn-default:hover,
.btn-default:focus {
background-color: #e0e0e0;
background-position: 0 -15px;
}
.btn-default:active,
.btn-default.active {
background-color: #e0e0e0;
border-color: #dbdbdb;
}
.btn-primary {
background-image: -webkit-linear-gradient(top, #3498db 0%, #2077b2 100%);
background-image: linear-gradient(to bottom, #3498db 0%, #2077b2 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff3498db', endColorstr='#ff2077b2', GradientType=0);
filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
background-repeat: repeat-x;
border-color: #1e72aa;
}
.btn-primary:hover,
.btn-primary:focus {
background-color: #2077b2;
background-position: 0 -15px;
}
.btn-primary:active,
.btn-primary.active {
background-color: #2077b2;
border-color: #1e72aa;
}
.btn-success {
background-image: -webkit-linear-gradient(top, #2ecc71 0%, #239a55 100%);
background-image: linear-gradient(to bottom, #2ecc71 0%, #239a55 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff2ecc71', endColorstr='#ff239a55', GradientType=0);
filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
background-repeat: repeat-x;
border-color: #219251;
}
.btn-success:hover,
.btn-success:focus {
background-color: #239a55;
background-position: 0 -15px;
}
.btn-success:active,
.btn-success.active {
background-color: #239a55;
border-color: #219251;
}
.btn-info {
background-image: -webkit-linear-gradient(top, #9b59b6 0%, #7a4092 100%);
background-image: linear-gradient(to bottom, #9b59b6 0%, #7a4092 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff9b59b6', endColorstr='#ff7a4092', GradientType=0);
filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
background-repeat: repeat-x;
border-color: #743d8b;
}
.btn-info:hover,
.btn-info:focus {
background-color: #7a4092;
background-position: 0 -15px;
}
.btn-info:active,
.btn-info.active {
background-color: #7a4092;
border-color: #743d8b;
}
.btn-warning {
background-image: -webkit-linear-gradient(top, #f1c40f 0%, #b8960b 100%);
background-image: linear-gradient(to bottom, #f1c40f 0%, #b8960b 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff1c40f', endColorstr='#ffb8960b', GradientType=0);
filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
background-repeat: repeat-x;
border-color: #ae8e0a;
}
.btn-warning:hover,
.btn-warning:focus {
background-color: #b8960b;
background-position: 0 -15px;
}
.btn-warning:active,
.btn-warning.active {
background-color: #b8960b;
border-color: #ae8e0a;
}
.btn-danger {
background-image: -webkit-linear-gradient(top, #e74c3c 0%, #cd2a19 100%);
background-image: linear-gradient(to bottom, #e74c3c 0%, #cd2a19 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffe74c3c', endColorstr='#ffcd2a19', GradientType=0);
filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
background-repeat: repeat-x;
border-color: #c42818;
}
.btn-danger:hover,
.btn-danger:focus {
background-color: #cd2a19;
background-position: 0 -15px;
}
.btn-danger:active,
.btn-danger.active {
background-color: #cd2a19;
border-color: #c42818;
}
.thumbnail,
.img-thumbnail {
-webkit-box-shadow: 0 1px 2px rgba(0, 0, 0, .075);
box-shadow: 0 1px 2px rgba(0, 0, 0, .075);
}
.dropdown-menu > li > a:hover,
.dropdown-menu > li > a:focus {
background-color: #e8e8e8;
background-image: -webkit-linear-gradient(top, #f5f5f5 0%, #e8e8e8 100%);
background-image: linear-gradient(to bottom, #f5f5f5 0%, #e8e8e8 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff5f5f5', endColorstr='#ffe8e8e8', GradientType=0);
background-repeat: repeat-x;
}
.dropdown-menu > .active > a,
.dropdown-menu > .active > a:hover,
.dropdown-menu > .active > a:focus {
background-color: #258cd1;
background-image: -webkit-linear-gradient(top, #3498db 0%, #258cd1 100%);
background-image: linear-gradient(to bottom, #3498db 0%, #258cd1 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff3498db', endColorstr='#ff258cd1', GradientType=0);
background-repeat: repeat-x;
}
.navbar-default {
background-image: -webkit-linear-gradient(top, #fff 0%, #f8f8f8 100%);
background-image: linear-gradient(to bottom, #fff 0%, #f8f8f8 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffffffff', endColorstr='#fff8f8f8', GradientType=0);
filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
background-repeat: repeat-x;
border-radius: 3px;
-webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, .15), 0 1px 5px rgba(0, 0, 0, .075);
box-shadow: inset 0 1px 0 rgba(255, 255, 255, .15), 0 1px 5px rgba(0, 0, 0, .075);
}
.navbar-default .navbar-nav > .active > a {
background-image: -webkit-linear-gradient(top, #ebebeb 0%, #f3f3f3 100%);
background-image: linear-gradient(to bottom, #ebebeb 0%, #f3f3f3 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffebebeb', endColorstr='#fff3f3f3', GradientType=0);
background-repeat: repeat-x;
-webkit-box-shadow: inset 0 3px 9px rgba(0, 0, 0, .075);
box-shadow: inset 0 3px 9px rgba(0, 0, 0, .075);
}
.navbar-brand,
.navbar-nav > li > a {
text-shadow: 0 1px 0 rgba(255, 255, 255, .25);
}
.navbar-inverse {
background-image: -webkit-linear-gradient(top, #3c3c3c 0%, #222 100%);
background-image: linear-gradient(to bottom, #3c3c3c 0%, #222 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff3c3c3c', endColorstr='#ff222222', GradientType=0);
filter: progid:DXImageTransform.Microsoft.gradient(enabled = false);
background-repeat: repeat-x;
}
.navbar-inverse .navbar-nav > .active > a {
background-image: -webkit-linear-gradient(top, #222 0%, #282828 100%);
background-image: linear-gradient(to bottom, #222 0%, #282828 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff222222', endColorstr='#ff282828', GradientType=0);
background-repeat: repeat-x;
-webkit-box-shadow: inset 0 3px 9px rgba(0, 0, 0, .25);
box-shadow: inset 0 3px 9px rgba(0, 0, 0, .25);
}
.navbar-inverse .navbar-brand,
.navbar-inverse .navbar-nav > li > a {
text-shadow: 0 -1px 0 rgba(0, 0, 0, .25);
}
.navbar-static-top,
.navbar-fixed-top,
.navbar-fixed-bottom {
border-radius: 0;
}
.alert {
text-shadow: 0 1px 0 rgba(255, 255, 255, .2);
-webkit-box-shadow: inset 0 1px 0 rgba(255, 255, 255, .25), 0 1px 2px rgba(0, 0, 0, .05);
box-shadow: inset 0 1px 0 rgba(255, 255, 255, .25), 0 1px 2px rgba(0, 0, 0, .05);
}
.alert-success {
background-image: -webkit-linear-gradient(top, #2ecc71 0%, #27ad60 100%);
background-image: linear-gradient(to bottom, #2ecc71 0%, #27ad60 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff2ecc71', endColorstr='#ff27ad60', GradientType=0);
background-repeat: repeat-x;
border-color: #208e4e;
}
.alert-info {
background-image: -webkit-linear-gradient(top, #9b59b6 0%, #8747a2 100%);
background-image: linear-gradient(to bottom, #9b59b6 0%, #8747a2 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff9b59b6', endColorstr='#ff8747a2', GradientType=0);
background-repeat: repeat-x;
border-color: #713b87;
}
.alert-warning {
background-image: -webkit-linear-gradient(top, #f1c40f 0%, #cea70c 100%);
background-image: linear-gradient(to bottom, #f1c40f 0%, #cea70c 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff1c40f', endColorstr='#ffcea70c', GradientType=0);
background-repeat: repeat-x;
border-color: #aa8a0a;
}
.alert-danger {
background-image: -webkit-linear-gradient(top, #e74c3c 0%, #e12e1c 100%);
background-image: linear-gradient(to bottom, #e74c3c 0%, #e12e1c 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffe74c3c', endColorstr='#ffe12e1c', GradientType=0);
background-repeat: repeat-x;
border-color: #bf2718;
}
.progress {
background-image: -webkit-linear-gradient(top, #ebebeb 0%, #f5f5f5 100%);
background-image: linear-gradient(to bottom, #ebebeb 0%, #f5f5f5 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffebebeb', endColorstr='#fff5f5f5', GradientType=0);
background-repeat: repeat-x;
}
.progress-bar {
background-image: -webkit-linear-gradient(top, #3498db 0%, #217dbb 100%);
background-image: linear-gradient(to bottom, #3498db 0%, #217dbb 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff3498db', endColorstr='#ff217dbb', GradientType=0);
background-repeat: repeat-x;
}
.progress-bar-success {
background-image: -webkit-linear-gradient(top, #2ecc71 0%, #25a25a 100%);
background-image: linear-gradient(to bottom, #2ecc71 0%, #25a25a 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff2ecc71', endColorstr='#ff25a25a', GradientType=0);
background-repeat: repeat-x;
}
.progress-bar-info {
background-image: -webkit-linear-gradient(top, #9b59b6 0%, #804399 100%);
background-image: linear-gradient(to bottom, #9b59b6 0%, #804399 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff9b59b6', endColorstr='#ff804399', GradientType=0);
background-repeat: repeat-x;
}
.progress-bar-warning {
background-image: -webkit-linear-gradient(top, #f1c40f 0%, #c29d0b 100%);
background-image: linear-gradient(to bottom, #f1c40f 0%, #c29d0b 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff1c40f', endColorstr='#ffc29d0b', GradientType=0);
background-repeat: repeat-x;
}
.progress-bar-danger {
background-image: -webkit-linear-gradient(top, #e74c3c 0%, #d62c1a 100%);
background-image: linear-gradient(to bottom, #e74c3c 0%, #d62c1a 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffe74c3c', endColorstr='#ffd62c1a', GradientType=0);
background-repeat: repeat-x;
}
.list-group {
border-radius: 3px;
-webkit-box-shadow: 0 1px 2px rgba(0, 0, 0, .075);
box-shadow: 0 1px 2px rgba(0, 0, 0, .075);
}
.list-group-item.active,
.list-group-item.active:hover,
.list-group-item.active:focus {
text-shadow: 0 -1px 0 #217dbb;
background-image: -webkit-linear-gradient(top, #3498db 0%, #2384c6 100%);
background-image: linear-gradient(to bottom, #3498db 0%, #2384c6 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff3498db', endColorstr='#ff2384c6', GradientType=0);
background-repeat: repeat-x;
border-color: #2384c6;
}
.panel {
-webkit-box-shadow: 0 1px 2px rgba(0, 0, 0, .05);
box-shadow: 0 1px 2px rgba(0, 0, 0, .05);
}
.panel-default > .panel-heading {
background-image: -webkit-linear-gradient(top, #f5f5f5 0%, #e8e8e8 100%);
background-image: linear-gradient(to bottom, #f5f5f5 0%, #e8e8e8 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff5f5f5', endColorstr='#ffe8e8e8', GradientType=0);
background-repeat: repeat-x;
}
.panel-primary > .panel-heading {
background-image: -webkit-linear-gradient(top, #3498db 0%, #258cd1 100%);
background-image: linear-gradient(to bottom, #3498db 0%, #258cd1 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff3498db', endColorstr='#ff258cd1', GradientType=0);
background-repeat: repeat-x;
}
.panel-success > .panel-heading {
background-image: -webkit-linear-gradient(top, #2ecc71 0%, #29b765 100%);
background-image: linear-gradient(to bottom, #2ecc71 0%, #29b765 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff2ecc71', endColorstr='#ff29b765', GradientType=0);
background-repeat: repeat-x;
}
.panel-info > .panel-heading {
background-image: -webkit-linear-gradient(top, #9b59b6 0%, #8f4bab 100%);
background-image: linear-gradient(to bottom, #9b59b6 0%, #8f4bab 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ff9b59b6', endColorstr='#ff8f4bab', GradientType=0);
background-repeat: repeat-x;
}
.panel-warning > .panel-heading {
background-image: -webkit-linear-gradient(top, #f1c40f 0%, #dab10d 100%);
background-image: linear-gradient(to bottom, #f1c40f 0%, #dab10d 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#fff1c40f', endColorstr='#ffdab10d', GradientType=0);
background-repeat: repeat-x;
}
.panel-danger > .panel-heading {
background-image: -webkit-linear-gradient(top, #e74c3c 0%, #e43725 100%);
background-image: linear-gradient(to bottom, #e74c3c 0%, #e43725 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffe74c3c', endColorstr='#ffe43725', GradientType=0);
background-repeat: repeat-x;
}
.well {
background-image: -webkit-linear-gradient(top, #e8e8e8 0%, #f5f5f5 100%);
background-image: linear-gradient(to bottom, #e8e8e8 0%, #f5f5f5 100%);
filter: progid:DXImageTransform.Microsoft.gradient(startColorstr='#ffe8e8e8', endColorstr='#fff5f5f5', GradientType=0);
background-repeat: repeat-x;
border-color: #dcdcdc;
-webkit-box-shadow: inset 0 1px 3px rgba(0, 0, 0, .05), 0 1px 0 rgba(255, 255, 255, .1);
box-shadow: inset 0 1px 3px rgba(0, 0, 0, .05), 0 1px 0 rgba(255, 255, 255, .1);
}
/*# sourceMappingURL=bootstrap-theme.css.map */

Some files were not shown because too many files have changed in this diff Show More