CoW - Completed Tasks
Note: add new completed tasks at the bottom (i.e. this is in chronological
order, oldest first).
- Created sins/versioning and adapted the SVNKit example code as a test suite.
Successfully creates repositories and does all the usual SVN stuff on them.
- Done using Appfuse 1 because 2 not ready yet
- Using Equinox or Appfuse 2 seed a webapp to host GATEWiki as part of the
sins tree. This should be a maven multi-project build incorporating the
webapp, yam and clone. Use Spring 2 / Spring MVC. Integrate with the
cruise control build/test.
- Choose a testing strategy, write some initial tests (including against the
test instance), add it to the Cruise Control suite (i.e. the top-level ant
build) and document the build/test arrangements in the
developer guide.
- Implement Spring/Acegi registration and login
- Done using Hypersonic and DAOs: Allow registration of sandboxes; store the
list in a ~/.sins.xml file.
- TAO and MC sites: move to gateservice1
- Reconfigure GATEWiki to use Jetty, and add a sins/gatewiki/jetty tree.
- Investigated using Tomcat Virtual Directory: It is possible to set that
pages are opened from different location and not only from Tomcat deploy
(webapps) directory. The effort is adding a new .xml file where the context
and a docBase is specified, so that Tomcat "knows" where to look under
specified context. Different types of files could be opened (.html, .pdf,
.jpg, .avi, .yam). Doesn't require Tomcat restart.
- Jetty: since configuration is done in jetty.xml requires restart.
- GATE movies: add to DD's set, and add to SVN
- Use log4j in gate.versioning
- [GATEWiki] SVNKit integration (see also gatewiki.html#bean-interactions):
- Enumerate functions needed from SVNKit (e.g. getting a list of what has
changed when updating) and implement example tests in sins/versioning.
Finalise the Sandbox object model.
- SandboxManager: singleton that allows getting of Sandbox but is the only
place Sandboxes are created; guarantees one Sandbox per disk path
- Sandbox: prototype bean created only by SandboxManager. All locking
operations must be synchronised and atomic.
- [YAM] Add logging and spring as in gate.versioning; iohandler use FSRs
- [YAM] deal with threading issues in Yam and make it a spring bean
- [GATEWiki]: Implemented representation of sandbox directory tree with Struts
Menu Tag Library and a util class to work with file system. This repository
tree is loaded during startup of app and set to be an attribute of
ServletContext. Some problems found with Jetty when this repository is of
bigger size (e.g. when I add current sins project to the registered
sandboxes list). Tomcat handles this fine.
- [YAM] implement new top-level API based on YamFile
- record generation runs and prevent concurrent identical translations
- implement naive "needsRegen"
- wiki links translated according to if they exist (if they don't then use
the URL of a "create" page)
- [GATEWiki]:
- Whenever asked to serve a PDF or HTML check for a more recent YAM and
regenerate if necessary.
- When requested, relative to a particular directory or file, do an SVN
update or commit, tracking the resultant changes and regenerating where
necessary.
- Dynamicaly passing username/password from gatewiki to versioning:
- Changed versioning method for adding sandboxes, so that now username and
password are passed to enable svn authentification.
- Changed gatewiki controller to pass the username and password when
adding Sandbox.
- SandboxModel table has now one more field which is reduntant but is
needed and that is 'username' which is actually a username of a user who
registered that sandbox.
- We are currently assuming that users use their databases (with list of
sandboxes only) localy - db is not shared. This means that only one user
will register all sandboxes.
- We also assume that svn username/password is the same for all registered
sandboxes.
- In order to user log in username/password for svn authentication, acegi
password encryption feature is turned off.
- [GATEWiki] experimented with xwiki
- [GATEWiki] Investigate Grails <HC>
- grails app for wiki
- grails 0.5 (head) install
- put in sins/clare svn
- add sandbox domain class
- made db persistent for development and production
- [YAM] Yammering <HC>
- make it possible to ignore titles in included files
- get tests working again
- add a test for title ignore during include (includes2.yam)
- add option and make it the default
- create doc/yam-syntax.yam, from yam-minimal.yam
- package.html etc.
- document the build/test process (include check-errors, bin/install)
- [CLONE] Improvements of CLONE <DD>
- Delete punctuation from the canonical value of quoted chunks.
- Unparseable sentences are silently ignored now: terrible for the users!
Work out how to apply rules in order with a final catch-all that returns a
good warning (investigate to see if there is a smart solution to do this
in JAPE).
- Investigate possible bug that a newline sometimes needs to follow the last
sentence (couldn't reproduce).
- Support for setting different datatypes (string, date, double). 'Projects
have date deadlines.' will create datatype property deadline with type
date. 'Projects have numeric budget.' will create double data type for new
property.
- Support for adding synonyms to classes not only to instances added.
- LabelIndex removed and Clie class refactored to handle methods that used
to be in LabelIndex class. This reduces bug re. refreshing labels after
introducing new ontology API.
- Changed the unit test to use Ontology.toString() method. This method
creates sorted string out of the ontology so that each time this method is
called for the same ontology it generates the same string.
- Make a text file to mention all possible combinations (sentences) that
could be proccessed with clie. This text file is used in tests now.
- Implemented web service for clone, and basic webapp that uses this ws to
answer user questions.
- [CLONE] Querying with CLONE <DD>
- Querying ontology with simple questions like: how many projects are there?
or what are the types of documents?/what is the deadline of sekt?/list
authors of 'some paper'. Look at query-example.txt for some samples.
- [CLONE]: separated query language application from clone. <DD>
- [CoW] Controllable Wiki <HC>
- creation
- using grails 0.5
- grails create-app cow
- adding to svn (http://grails.codehaus.org/Checking+Projects+into+SVN):
- svn add cow; svn ci -m added cow; cd cow
- svn propset svn:ignore "WEB-INF" web-app/
- svn propset svn:ignore "core" plugins/
- svn rm web-app/WEB-INF
- svn rm plugins/core
- svn commit -m "Fixing SVN"
- grails upgrade
(this re-seeds the non-versioned stuff)
- sandboxes
- added sandbox domain class
- to get default controller, view etc. used:
- def scaffold = Sandbox in SandboxController
- validated sandbox filename
- persistence settings
- made db persistent for development and production
- persistent data: edit grails-app/conf/DevelopmentDataSource.groovy to
set HSQLDB to "file" and dbCreate to "update"
- seems to have a 20 second write delay
- build
- changed the ant build to cater for the new stuff
- moved ~/.m2 into sins; now versioning and yam take their gate.jar
dependency from sins/maven-repository (so that we can remove the need
for installing any maven-related stuff in order to build, e.g. with CC)
(see sins/build.xml)
- upgraded gate jar to 4.0; installed it to local rep using:
- mvn install:install-file -DgeneratePom=true
- -Dmaven.repo.local=/home/hamish/sins/maven-repository
- -Dfile=/home/hamish/gate/bin/gate.jar -DgroupId=gate
- -DartifactId=gate -Dversion=4.0beta1 -Dpackaging=jar
- [CoW] CoW: Controllable Wiki <HC>
- general
- did grails install-templates: creates src/templates
- serving static pages as a component of dynamic pages.
- used generate-all to get the controller and view code
- played with sitemesh api and a view to render a static page
- investigated the problems with url mappings
- implemented a UrlRewriteFilter to change url paths to ? syntax
- see also notes in gatewiki.html
- upgraded to grails HEAD (0.5.5) and re-test
- fixed startup .zip problem: delete .tar etc. in grails/dist
- [CoW] CoW: Controllable Wiki <HC>
- serving static pages
- changed the views so that links to pages ending in id numbers have a
trailing /
- changed "sandbox" to "wiki" with an action "area" so URLs look like
/cow/wiki/area/4?path=...
- added urlrewrite jar and WEB-INF/web.template.xml to SVN
- found sitemesh bug on gate.ac.uk index.html and wrote workaround
(gate.util.GFiles)
- using NanoHTTPD.serveFile to construct responses for non-HTML and
directory files
- could in future use java activation to guess mime type and so on; see
dani's code, e.g.: def fds = new
javax.activation.FileDataSource(fileName);
- made changes to NanoHTTPD to remove the / from the paths it returns
- if contentType of response is set to "text/html" grails incorrectly
tries to construct a view...
- finally fixed the paths issues and got static files working properly
- images now working but not in subdirs because when including the body
of a page in a subdir, the relative links will inevitably be broken...
- tried adding the path part of the ?page= spec, but of course this
is not set for the relative links :-(
- options:
- go back to the hacked UrlMappingsFilter now the id/ thing has solved
part (all?) of the problems that it had? (worked!)
- clean test in grails
- implement filter hack and re-test
- if succeeds try in cow
- turn off url rewrite filter
- change cow to use CowUrlMappings again
- submit JIRA with the patch
- redirect the *.* urls and use CowUrlMappings to process the dir
parts? (rejected)
- misc
- css etc.:
- modify src/templates before generating controllers and views
- logo: grails-app/views/layouts/main.gsp link, and put the .png in
web-app/images
- [CoW] CoW: Controllable Wiki <HC>
- cleaned up static code and removed NanoHTTPD dependency
- serving static pages
- delete sandbox
- split WikiController to create WikiAdminController, and move the
relevant views
- add dirListingAllowed to Wiki
- views: change wiki show to only show id
- added jetty to sins and checked deployment
- added new actions and menus, and a top-level, and a short description to
Wiki
- install versioning bean into cow (just add the spring config and make it
a member of a controller and it should be autowired) (uses sb mgr for
serialisation of access to dir trees)
- first version of page edit action with YAM summary
- designed layout and made some examples for the CSS stuff
- documented the build process
- [CLOnE]
- Created an ontology lookup component [DD, VT]
- Created an application integrating a Tokeniser, SentenceSplitter,
POSTagger, Morph and the above mentioned ontology lookup component that
annotates an input document with mentions of ontological resource (Using
flexible gazetteer) [DD, VT]
- Developed a scoring strategy to combine specificity of the property and
also the similarity of its name with existing properties [DD, VT]
- [CoW] CoW: Controllable Wiki <HC>
- added WikiPage class in WikiController to simplify action logic
- fix bug with markup in .yams and filling text area for edit (use textarea)
- added contextPath stuff in YamFile and AbstractTranslator to support
translation in a different directory to the rest of the sources
- finished preview (uses show view with preview set true and temp file
POSTed on to save from the save link)
- finished complete edit/preview/save cycle (save gets a POST with the temp
file name that contains the new version of the file, and, in workstation
mode, just stuffs it on disk, calls YamFile.translate and redirects to
show with a flash message)
- cleaned up save code and added more checks on WikiPage in WC actions
- added create page / make directory
- [CoW] CoW: Controllable Wiki <HC>
- added dir list option to all html show views
- renamed WikiAdminController as AdminController
- [YAM] Yammering <HC>
- added gate.yam.dependencies to sins/yam in groovy and get the ant build to
add it to the YAM snapshot jar
- reverted to ant and cleaned up the build.xml
- [CoW] CoW: Controllable Wiki <HC>
- added Canoo functional tests
- tested admin, wiki page creation and edit, including the svn results
- new create stuff
- added create new page and linked from directory list views
- create of existing forwards to show
- create of non-html/non-dir messages "use upload"
- [CoW] security, authorisation and authentication (see also
gatewiki.html#section-4.3.)
- Acegi based security prototype <JS>
- Send a Acegi plugin patch to Grails list (support M:N relation between
authorities and request maps) IN PROGRESS
- Add initial data somehow, instead of invoking setup controller. In
ApplicationBootStrap should be figured out if the DB is empty and
populate it with some data. Details, welcome. (If you do Wiki.list and
get nothing, that indicates that the DB contains no wikis, for example.)
- Person - Authority. Add custom code on scaffolded GSP pages:
- add/update person - display checkboxes for available authorities
- delete authority - display warning that there are users with that role
and that they should be edited first
- Wiki - RequestMap. Implement 1:N relationship with cascade delete.
- add/update wiki - display matrix ROLE name : R/W privileges. After
submit, delete all request maps for particular wiki and add new ones
in the way it works now
- when you delete wiki, delete all request maps that belong to that wiki
- do not provide Edit Request Map, since it has no sense. Request map
lives as long as Wiki lives, so we should edit Wiki only
- Add access denied page. Now Acegi plugin cannot find it in case that
user is not allowed to see particular URL, therefore Error 404 occurs.
- Use existing grails gsp tags to show Login/Logout links depending on if
the user is authenticated or not. Details, welcome. (try the acegi
taglib and g:if as needed)
- Use existing grails gsp tag to find authority of authenticated user and
display him particular link (e,g Admin in header). Details, welcome.
- [Versioning] Versioning API <HC>
- changed the API to use File instead of FileSystemResource, because FSR
doesn't add anything for normal use, and are a pain to get (requiring a
Spring context)
- [CoW] security, authorisation and authentication <JS, HC>
- integration log: followed james' instructions except:
- didn't copy cow-acegi/views/wikiAdmin/list to cow/views/admin/show as it
doesn't seem to be different from the old one
- didn't copy the top bar toggle javascript stuff from main.gsp to
wiki.gsp (although did copy to main.gsp, which now just serves the admin
controller views) as james will integrate this directly with the trunk,
and it needs adapting to the new cow CSS layout
- didn't copy the main.css as the main changes there seem to support the
top bar toggle stuff... james overwrote it
- omitted welcome message, as I intend that the user name will be a link
to an account management page
- fixed integration issues
- in w/s mode acegi stuff should do nothing.
(-Dgate.cow.mode=workstation)
- changes to names of classes and fields (where these are from the Acegi
Plugin code you can leave the names in the code the same but change them
in the GSPs)
- Requestmap to RequestMap
- Emailshow to ShowEmail
- Passwd to PassWord
- Group to Role
- ConfigAttribute to Roles
- RequestMap URL to URI
- UserName to LoginName
- UserRealName to FirstName plus LastName
- Authority to Role
- UI supporting anon read (not just "don't set any readers").
- just display the authority name, not the ROLE_... name
- fixed acegi filter prob in workstation mode
- fixed: the newpage redirect to directory incorrectly strips the trailing /
from the requestPath (HC)
- reorganised the admin areas and / page
- [CoW] build, restructuring <HC>
- moved yam to cow
- src/java/resources gets put in lib/cow-resources.jar
- pick up yam/bin stuff
- made the ant build copy the test results for workstation and server mode
and link from cow/index
- finished war, deploy, etc. and sins/build.xml
- clean checkout/cruise testing
- add the clean shutdown option to run-dev too
- [CoW] config, DBs and other user data: CowDataArea (CDA) <HC>
- bootup: use doWithSpring to init the CDA
- ApplicationBootstrap: if no Wikis exist create 1 on help and 2 on main
- [CoW] converters <VT>
- JSPWiki2Yam converter
- HTML converter
- [CoW] grails 0.6
- moved CowUrlMappings to UrlMappings
- moved datasource stuff to DataSource
- moved log4j to Config
- updated the build
- acegi
- fixed doWithSpring to replace cf method
- commented out non-needed resources from acegi plugin.xml
- changed the one line of AcegiGrailsPlugin that didn't use
transformToGetter
- [CoW] grails 1.0-RC1
- upgraded libraries to match grails versions
- [CoW] sourceforge release
- set up new sourceforge project and moved the code there
- helpdocs.zip moved to src/java/resources so that can find them in the war
deploy
- create new page in wiki root fixed
- [CoW] tests
- webtest didn't work since canoo moved to a plugin
- [CoW] security, authorisation and authentication <JS, HC>
- add user-level account creation, with
- [CoW] security, authorisation and authentication <JS>
- upgrade to acegi 0.2
- webtests for crud over acegi, user registration
- [CoW] security <HC>
- to prevent embedded HTML or JavaScript attacks in wiki pages, the YAM
translator needs to call
org.springframework.web.util.HtmlUtils.htmlEscape(text.toString()) on all
non-control text (ASTPlain, ASTVerbatim and ASTUrl)
- [Cow] Update Aceg-plugin to 0.2 <JS>
- removed old acegi-plugin 0.1 and captcha-0.5 plugin(as acegi-plugin now
includes it)
- New features of acegi-plugin 0.2 include some scripts to generate custom
stuff:
- grails create auth-domains User Role, which will create acegi domain
classes into cow/grails-app/domain/ and local AcegiConfig file into
cow/grails/conf
- grails create-requestmap, which will generate Requestmap domain.
- set loginUserDomainClass "User" and authorityDomainClass"Role" in the
AcegiConfig
- grails generate-manager, which generates all the controllers and views
for acegi stuff.
- grails generate-registration, which generate the controller and views
for registration.
- added webtest for User, Role, Requestmap and register etc.
- modified the TestSuite to ask it to load the tests orderly as
RequestmapTest must run before AdminTest
- updated web-app/plugins/acegi-0.1 to acegi-0.2
- [CoW] tests <JS>
- user and registration-related webtests should only run in server mode
- [CoW] additional wiki functions (e.g. wysiwyg edit, delete, rename)
- wysywig editing
- currently it seems to delete the title; is this a converter problem?
- tried fck editor plugin, but buggy
- try cleaning, removing web-app and plugins versions of fck
- add fck in web-app/js (2.4.3) and FckEditorTagLib from 0.5.6
- try richTextEditor tag; works, use this instead of plugin to avoid
plugin bugs and simplify maintenance
- add tests for the html2yam translator and define the necessary fixes to
the yam2html xslt for valy
- changes to the HtmlToYamConverter
- html generation <HC>
- rework the citation span to include the ids of the links: the cite
key.
- comments are lost, add cow-comment class
- put span class cow-footnote-text plus id around footnote text and add
the id to the footnote reference
- put a class on the html element cow-html. comment: this is not valid
html. meta tag added instead.
- XSL/converter changes <VT>
- deal with footnotes
- image attributes aren't translated, e.g. '%image(test-image.png,
ALT tag, 200, 200, left, 0)' becomes %image(test-image.png, ALT tag)
instead of %image(test-image.png, ALT tag, 200, 200, left, 0)
- it adds a new line before each list
- it loses newlines within lists, but seemingly not in ordered lists?
(not explicitly fixed, but can't replicate - perhaps dissapeared with
other changes?)
- the processEscapes methods should not rely on cow-escape but look for
class cow-html on the html element (low priority - I've turned these
methods off at present). Methods removed.
- convert nbsp to %\
- recognise citation URLs (span class cow-citation) and convert them to
%cite(...) notation
- recognise comments and convert to YAM notation
- fix citation hardcoding <AR>
- big.bib and citation links are hardcoded. Remove this, setting on
YamFile from application config.
- [CoW] various code cleanups <JS>
- Admin.save, Admin.update: factor out the common code into a method
- put all the auth-related code from CoW that's not in the acegi plugin into
a class
- migrate the path analysis stuff to **:
http://jira.codehaus.org/browse/GRAILS-1428
- move cow-test-area to "cow test area" to trap spaces in paths bugs
- [CoW] security bugs <JS>
- fix exception when logging in with wrong password (add to webtests)
- [CoW] security etc. <JS>
- all the acegi-related pages are now mapped in workstation mode to an
"unavailable page"
- login link should not show up in workstation mode (change to plugin
isNotLoggedIn tag)
- [CoW] build <VT>
- put the normal classes dir on the classpath and make java-tests-compile
depend on java-compile
- copy non-java files from the source path to the classes dir to remove the
need for the sourcepath on the classpath (which would break the war) for
finding .xsl etc.
- [YAM] Yammering <HC>
- contents get wrapped in paragraphs, which results in empty tags in the
html. solution: move contents under Unit instead of Paragraph
- [CoW] wysywig editing
- changes to the HtmlToYamConverter <VT>
- html generation
- div shouldn't be in a p (contents)
- classes:
- identify the title-derived headers and put class =
cow-title-heading
- footnote sub and links class = cow-footnote
- auto-generated section anchors class = cow-section-anchor
- escapes class = cow-escape
- add a class to citation URLs (cow-citation with id = to the cite
keys)
- when doing FCK edit run YF.generate with no include processing; document
- XSL changes
- don't add backslashes in front of dashes unless they are at the start
of a line preceded only by whitespace
- put a space after dashes derived from li
- don't put any blank lines before the title
- don't copy the title as a first level heading
- don't add a blank line at the start of tables
- table rows must be separated by "---"
- don't add a space before table column separators
- when processing "h1" etc., look for a span of class "cow-sec-number"
within the heading. if it exists, discard that span and put "%1" etc
as at present. if it doesn't exist, put "%1*" etc.
- when there's a cow-contents div, replace it with %contents
- don't generate a --- after the last table row
- ordered lists should start in "#" not "o"
- don't escape e.g. the % in %include; in fact don't do escaping at all
for now and we'll see how much of a pain this proves to be (maybe
none)
- process STRONG like B
- note: to generate diffs from some sample .yams try "ant
-Djava.test.case=gate.yam.YamTest java-test" and look at the ...-fault
file in comparison with the .yam in test/resources/yam
- [YAM] Yammering <HC>
- added lists of links and includes to YamParseTree (to support YamFile
dependency tracking)
- [YAM, CoW] dependency regeneration <AR>
- gate.yam.depend.Dependencies API:
- testing cow hookup / fix bug with file endings (Deps was expecting link
file suffixes to be in the wrong form, e.g. .yam or no suffix or
something. Links should be .html.
- See also the manual.
- includes(YamFile, List<String>): update the dependency graph with the
fact that YF includes a list of files
- linkedBy(YF, List<String>): ditto for linking to
- setStorageDir(File): where to save its serialised form
- needs to use Concurrent data structures
- YamFile: add new API to support dependency regeneration and to move some
YAM-specific stuff out of WikiController
- has a Dependencies member (set by clients eg. WikiController, one per
Wiki; may be null)
- calls Dependencies methods at each generate, delete, etc.
- new methods:
- getOutputLocations: get a list of outputs (tex, pdf, html)
- (existing) generate - ensure only single generate of any particular
YAM output
- generateDependencies: regenerate all dependent out-of-date files
- delete: delete .yam and outputs
- create: create empty.yam and generate
- rename: rename .yam and outputs
- wasModified: update depedencies for a modification (edit)
- WikiController:
- refactor so that YamFile does create/rename/delete etc.
- later on: gets upload, delete, SVN up etc. also send events to
Dependencies
- serialises Dependencies into ~/.cow (need to add config stuff now)
- gate.yam.depend.Dependencies API:
- Dependencies' main role is to let YamFile know which .yams need
regenerating whenever YamFile gets a change event (wasModified, create,
delete, rename). This probably means that it needs methods for all these
events that return a list of out-dated (dirty) files.
- The call sequence then goes like this:
- when WikiController is asked to save an edit it calls
YamFile.wasModified
- YF calls Dependencies.wasModified(this.file), which returns a list
of all .yams which include YF (that being the set which now need
regenerating due to the edit on YF)
- YF then does the regeneration of all the dirty dependencies
- Therefore Dependencies needs new methods for each event, taking a File
in each case.
- I haven't got my head properly around the code in Dependencies now,
but the above implies that the indices (map keys) need to be the ids
(canoncical paths) of files that are linked to or includes (i.e. the
indices need to answer the question "which are all the files that link
to this id?")
- Why do these methods take File instead of YamFile? Because we also need
to keep dependency data for non-YAM files that are uploaded or deleted.
(Actually in the case of inclusion only YamFiles are relevant, but for
linking any file or directory can be relevant.) A YAM which links to a
non-YAM file needs to be regenerated if that file is created (uploaded),
deleted or renamed, for example.
- In Dependencies.clear probably you only need to call clear() on the
existing map, not create a new one? If you do have to create a new one,
then don't bother clearing the old one first?
- Dependencies and synchronisation
- Webapps have to serve multiple concurrent requests from browsers. This
means that any data which is shared across requests has to worry about
concurrent modification. This applies to the static members of
Dependencies, which are shared across the whole application.
- One way to deal with this is to make all the methods of the class that
manipulate shared data synchronised (though some attention has to be
paid to the semantics of static sync methods vs. those of instance
methods). Another is to have syncronized blocks within methods that
reference the particular member objects that relevant during that
block of code.
- This stuff is pretty difficult to get right, so ideally we need a
comprehensive test for this stuff. (In fact for the whole thing, as we
don't have any multithreaded tests at present; probably there are
libraries out there that can help with this type of testing.)
- Because there are some pathological cases for regeneration (e.g. a file
which includes all others will need regeneration whenever any change is
made anywhere) we will probably need to implement some kind of queue for
regeneration requests.
- still got the modify issue? When a file is created, modify is called.
Unable to replicate. Possibly, user was creating via a create link, and
then editing the created file - so create then modify do get called.
- change WC/create to only do one call to Deps for create. Originally, WC
called create for the yam file, and create for the html file generated
from the yam. But Deps only stores dependencies for the yam. Cannot
replicate: must have been changed at some point. <AR>
- write the WikiController and Wiki code to start hooking up the new
Dependencies stuff and give to Angus <HC>
- generate and regenerate
- issues to deal with later:
- Quartz job to serialise Dependencies? Once per minute? <AR>
- [CoW, YAM] bugs
- bug 1819990
- fails to find body end tag in gate-server/html/demos/movies.html
- cannot replicate
- adopt ucu/MungeBanner to replace the page parsing stuff in WikiPage or
wherever it is
- look at using tag soup as a neko fallback
- /home/hamish/public_html/gau-stats-before-move.html leads to out of
memory error, probably because of some fault in HTML parsing
- getBody index out of bounds exception on
http://localhost:8080/cow/wiki/show/2/house/loft-2005/arthur-jan05.html
- Cannot replicate - this is an empty html file, which now gives an
erroneous yam file containing a single line break
- bug 1819991 (fixed)
- bug 1819994 (won't fix)
- there's probably a bug in ant target ordering that means that when ant
doc has never been called a webtest-workstation fails; also have to run
ant webtest at least once before can do ant run-dev after ant clean
- bug 1819993 (moved to backlog)
- there is no way to cancel a create, i.e. if you click on a create link,
the relevant file or directory is created and there's no way to back out
- bug 1838822 <JS>: fixed
- create new wiki (admin controller)
- the path should be validated: allow any letter, number, dash, dot, or
underscore
- when a path is entered that doesn't point to valid SVN sandbox there's
an exception; there should be a message "${path} is not a valid
Subversion sandbox" and the create new wiki page redisplayed. the Wiki
DB record should not be created
- bug 1819995: moved to backlog
- image: height should come last so can spec just width and also position?
or will there be some other general mech for positioning?
- bug 1838827 <JS> fixed
- "help" and "login" links should not be displayed on login page
- the "Admin" link should not be displayed on /admin pages (e.g.
/admin/list)
- bug 1819992 <JS> fixed
- on windows it is possible to create a non-canonical path (without the
drive letter)
- bug 1838819 <JS>
- the "new page name" box should have a tooltip saying "Enter a new page
name (e.g. 'new-page.html') or a new directory name (e.g. 'newdir').
For other page types use 'upload'. Note that names can't contain
spaces, apostrophes etc." This text should be in a message bundle (not
hard-coded).
- [CoW] misc
- DataSource doesn't get the DataArea stuff; load app.properties and get the
version from there
- change the "preview" flag to "p" to match the others
- moved YamHelp to YamService
- [CoW] testing
- added a webtest-quick that runs against an existing server
- if upgrading the webtest plugin note that there are patches against the
plugin directory for:
- copying the resources and making the links non-absolute (from jpv)
- look for a webtest.rungrails property and not to run the server if it
isn't true
- some small style changes
- [CoW] config, DBs and other user data
- added gate.cow configuration in Config and replaced DataArea etc. system
properties stuff
- [CoW] gate.ac.uk/wiki <HC, AR>
- JSPWiki2YAM: change o to #
- JSPWiki2YAM: try converting existing wiki and test <AR>
- Problems:
- A single underscore in jspw is a valid non-formatting character, but
starts an italic in yam. The code that escapes special characters in
the translation has been turned off with if(true) return,
deliberately. Fix in JSPWikiToYamConverter. Fixed.
- Headings in jspw do not have to be followed by a blank line, but do in
yam - undocumented syntax? Blank line needs adding in translation.
Fixed.
- Headings containing a $ get treated as if they contain a capture
group, leading to an error. Code uses regexes via replaceAll, when
should probably use Strings via replace. Fixed.
- Need to escape * and = in jspw in some or all cases. Fixed.
- Titles: JSPWiki has no notion of title within its syntax, although all
pages do have a title of the page name placed at the top (but this is
not defined in syntax). However, users sometimes put in a title of
their own at level 1. And sometimes they don't. YAM assumes the first
para is the title. If the user has placed a heading with a level as
the first para, then this gives e.g. %2* Some Heading. If the user
has placed no heading, and assumes the JSPW page title will act as
one, then they get their first content para as a title in YAM. Current
planned fix: blanket title of the page name. Fixed.
- Relative links have "VIEW" prepended, and are replaced with "create"
links. What is adding the "VIEW"? Fixed.
- Valentin.html; JapeHelp.html etc: < and > not translating. angle
brackets are perhaps assumed to be elements in jspw, and not rendered.
users therefore have to use < etc. Fixed.
- GateWiki.html link to GateDemoMovies not working: case-misnamed in
JSPWiki working copy and SVN. Was called GATEDemoMovies, but renamed
and not checked in. Fixed.
- teletype turned on part way through SourceForge.html: failing to place
the url in bracketed form. The person writing the JSPWiki page
intended the URL to be an unlinked, teletyped string. YAM will assume
it is for linking, and also try to teletype it. Tricky to sort out in
code, fixed by editing the offending wiki page. So it might happen
again on some future page... Fixed.
- GateWiki.html: faulty (and unneccesary?) link to an admin page.
JSPWikiMarkupParser makes a mess of this link, escaping the element <
and > surrounds. Remove such admin links. Fixed.
- Links containing commas don't convert. GLEAM.html: faulty "AJAX and
Web Services" link; Eclipse plugin external link broken on
Eclipse.html page. Escape commas and spaces in bracketed URLs. Fixed.
- encoding problem in MultiSourceIE.html ("Who's who"); SemanticWebTerms
("document's author"); InformationExtractionPapers (lots) etc etc.
There is a problem with JSPWiki txt files encoding, with some
characters being encoded and displayed as control characters. This is
most likely because someone copied and pasted out of a pdf document or
something similar. We can't recover the original characters. The most
faithful conversion is with input encoded as ISO-8859-1 and output as
UTF-8, which gives the same incorrect encodings in JSPWiki and yam.
- attachments: e.g. to GateDemoMovies.html. Need to get attachments from
similarly named directory, copy to output, and add to a link section
at the bottom of the page. Done.
- grails 1.0
- upgrade svnkit.jar (1.1.2 in grails RC3)
- edit link not showing at top level of wiki areas? (fixed in
PageController)
- notes on old edit flow (pre-PageController)
- views:
- layouts/wiki.gsp (used by show): <g:if test="${editAllowed &&
editLink}"> <span class="pageLink"> <g:link action="edit"
id="${editLink}">Edit this page</g:link> </span> </g:if>
- views/wiki/edit.gsp <g:form controller="wiki" action="preview"
id="${previewLink}" method="post"> <span
class="button"><g:actionSubmit value="Preview" /></span> <textarea
style="width:100%;" class="pageText" id="editorarea"
name="editedtext" rows="25" cols="80">${pageText}</textarea>
</g:form>
- views/wiki/show.gsp preview stuff <g:if test="${preview}"> <g:form
controller="wiki" action="save" id="${saveLink}" method="post">
<input type="hidden" name="editedTextPath"
value="${editedTextPath}" /> <input type="hidden" name="saveLink"
value="${saveLink}" /> <span class="button"><g:actionSubmit
action="save" value="Save this edit" /></span> </g:form> </g:if>
- WikiController
- show...edit...preview...show...save
- show puts editAllowed and editLink, the latter points to the id and
request path, and is used to point to the edit action by wiki.gsp
- edit puts previewLink
(${wiki.id}${wikiPage.requestPath}?preview=true) and pageText and
edit.gsp uses the link to point to preview from an edit form
- preview regenerates the HTML, and puts: editedTextPath:tempFilePath,
saveLink:"${wiki.id}${requestPath}" then renders the show.gsp view
with a form forwarding to save with the file path
- save overwrites the wiki files from the temp files, puts a message in
flash scope and redirects to show
- links
- [CoW, YAM] bugs
- bug 1819990
- fixed in PageController and FileUtils
- relative links that go outside of the current wiki area generate an
exception (because it tries to convert a non-int for the wiki id)
- coded, but needs tests: doesn't pick up charset information from headers
of static pages (will perhaps need to find the encoding from the doc
before processing into the dom).
- [CoW] PageController <HC>
- webflow exception serving binaries: moving it back into the controller
instead of the gsp and using the 1.0.1 snapshot removed the problem
- can't edit directory index pages implicitly
- rich yam edits remove the title: change to xsl translator
- html edits lose the header
- [YAM, CoW] dependency regeneration <AR, HC>
- generate and regenerate
- to begin with just execute the regenerate queue requests immediately
- server mode edits: ~/.cowrc.d/staging/$session.user ?
- first move all the saving stuff into YamService
- in workstation mode save just overwrites the sandbox file and does
generate
- in server mode, needs to use a staging area, and when saving update
(if conflict redirect back to edit with a flash message), log,
overwrite, commit, unlock
- issues to deal with later:
- priority queue for regenerate requests? <HC>
- Quartz job done, needs test<AR>
- YamFile.needsRegeneration will become redundant? <HC>
- [CoW] additional wiki functions (e.g. wysiwyg edit, delete, rename) <HC>
- wysywig editing
- disable uploads, pictures etc. in fckeditor
- [CoW] additional wiki functions (e.g. wysiwyg edit, delete, rename) <AR>
- changes to the HtmlToYamConverter
- XSL/converter changes <AR>
- linebreaks
- it adds spaces / new lines before the title
- Also too many added after titles, and in other places
- Possible problem with newlines and lists
- it loses newlines within URL texts (this is probably the fault of
the parser, and maybe can't be fixed easily)
- [gate.ac.uk] JSPWiki2YAM: try converting existing wiki and test <AR>
- Problems:
- TaoProjectWiki.html contains leading and unmatched [[ for some reason?
These have been placed there by the author. They are ignored by JSPWiki,
but picked up by yam. solution: edit the JSPWiki page.
- = is being incorrectly escaped in verbatim blocks
- Several config and system pages created, that we probably don't need
e.g. VariableTag.html, SystemInfo.html
- TaoProjectWiki.html is linked from GateWiki.html. It should, however, be
restricted to some users only. JSPWIki gives a "forbidden" message.
- Although attachments are now added, links to attachments in the text of
the page (as opposed to the list at the end) may need to be adjusted -
check.
- grep for broken links (localhost/create) and check them.
- Load into a runnig CoW
- dir listings: hide .* and *.yam by default, and have a config option or
a "show hidden" link to show them
- [CoW] PageController <HC>
- couldn't duplicate the gatewiki conversion probs that Angus couldn't
find... my error presumably
- regen: just execute the YS regenerate queue requests immediately
- [CoW] security <HC>
- tmp file paths should include some prefix (to prevent an attack that
downloads some known resident of /tmp) or put them into ~/.cowrc.d
- [CoW] gate.ac.uk/wiki <AR>
- XSL/converter changes: deal with bare predicates
- Check that dependencies is working properly when using PageController
(doesn't seem to queue anything? or poss I got the example wrong; it now
gets called from YamService.generate)
- [CoW]
- upgraded to grails 1.0.1 (6715) from 6688 snapshot (seems to create more
copies of services so had to change YamServiceTests)
- added a default index page in area 2 when created via DataArea
- [CoW] tests
- fixed the PageTest stuff to use a cow-local .cowrc etc. by setting
user.home to ./webtest/user-home
- changed webtest-quick to webtest-page and made it run irrespective of if
grails is running
- added webtest-page to cruise
- put "1 2 3" etc. in doc/test-resources
- added cow/dev-user-home and made run-dev use that
- [CoW] PageController etc. <HC>
- enumerate the redirects and GSP links for PC/PS and create a taglib
- layouts/page.gsp replaces wiki.gsp
- create URI is now not special; a 404 that's not a bad wiki area is trapped
by PC.show and redirected to the new page flow
- [CoW] Build
- move the various resources into src/java/gate, grails-app/utils/gate/cow
and test/unit/gate (these will be copied into the ~/.grails classes
directory)
- updating config to be like latest grails apps: delete the
grails.war.dependencies in Config (hard to maintain, doesn't contribute
much)
- Malformed refresh string (found ';' but no 'url/='):
0;/cow/page/show;jsessionid=ihit6iwer6ey/1/doc/
- [CoW] replacing Acegi with JSecurity, cleaning up
- branched to jsec-gatewiki
- delete (or move to doc/old for future ref)
- delete WikiController, AdminController, views; generate-all Wiki
- delete acegi, related controllers and views
- delete related webtests
- delete web-app/*.gsp
- moved to doc/old: plugins/grails-acegi-0.2.zip plugins/acegi-0.2
webtest/tests/RequestmapTest.groovy webtest/tests/RoleTest.groovy
webtest/tests/Login.groovy webtest/tests/UserTest.groovy
webtest/tests/WikiTest.groovy webtest/tests/RegisterTest.groovy
web-app/plugins/acegi-0.2 web-app/workstation-mode.gsp
grails-app/views/wiki grails-app/views/register grails-app/views/admin
grails-app/views/user grails-app/views/requestmap grails-app/views/role
grails-app/views/layouts/wiki.gsp
grails-app/controllers/RoleController.groovy
grails-app/controllers/WikiController.groovy
grails-app/controllers/AcegiHelper.groovy
grails-app/controllers/UserController.groovy
grails-app/controllers/AdminController.groovy
grails-app/controllers/CaptchaController.groovy
grails-app/controllers/RequestmapController.groovy
grails-app/controllers/RegisterController.groovy
grails-app/controllers/WikiPage.groovy
grails-app/conf/AcegiConfig.groovy grails-app/domain/Requestmap.groovy
grails-app/domain/Role.groovy grails-app/domain/User.groovy
- Dependencies: generate-all from Wiki puts the following into create.gsp
(which is invalid):
<g:select optionKey="id" from="${Linked by.list()}"
name="dependencies.id" value="${wiki?.dependencies?.id}" ></g:select>
- Remove the getDependencies (and wikiURI) methods from Wiki
- build: put grails lib on build classpaths and removed redundant libraries
from lib (all that are in grails/lib in fact); for some reason the
webtests couldn't then see svnkit, so I left that one in lib
- [CoW] cleaning up, new security model: jsec-gatewiki branch
- use pageURI to create links in PC and PS, and alter the taglib to be
sensitive to implicitID
- add in the 1 and 2 mappings and debug
- a little more PageTest... note that it does a hacky svn revert afterwards
- merged branch back into trunk
- [CoW] YAM <HC>
- change YamCommand to set createPage to ""
- [CoW] PageController <HC>
- renamed CowLinkTagLib to PageLinkTagLib
- errors: UrlMappings: "500"(view:'/error'), and make that cow style
- [CoW] security <AR>
- new version of jsec and debug - done
- permissions checks in the template GSPs and/or the taglib [document how
this now works] - done, see docs.
- presumably we need to encodeAsHtml whatever the user puts into the
form-based page edit if we want to prevent them embedding javascript? or
will yam2html sufficiently mangle it anyhow? Done - encodeAsHtml added to
edit.gsp
- check all GSP and controller rendering to ensure that encodeAsURL and
encodeAsHTML are used whenever there is (or could be) user-supplied data -
Done.
- [CoW] <TH>
- SVN integration: evaluate 3rd party repository browsing webapps
- check/update the bug tracker and other bugs below
- [CoW] <TH>
- changed the help link to point at the user manual
- [CoW] security <AR>
- A user with ReadAndWrite permissions on a wiki can edit the actual wiki
object itself. e.g. create a user with ReadAndWrite on a new area (4 in
this case) and then try: http://localhost:8080/wiki/show/4
- build into controllers: safeguards against deleting controllers that
are used in role
Old rejected stuff
- File system watching to do updates
- Watch the registered sandboxes using a low-priority thread that doesn't
hog machine resources. The point is to spot YAM files that have changed
more recently than the .html generated from them and queue them for
regeneration.
- Consume out-of-date YAM files from the queue and translate them using
gate.yam.Yam
- Making YamFile a spring bean: it is hard to do conditional construction of
these, and the use case of substituting a different implementation in XML
config doesn't apply here.
- Using Tomcat or Jetty virtual directories for serving registered sandboxes.
It will need some investigation in Jetty code, and also, the separate
impementation for different Web container. Additionally, some implemenations
will be essential in order to serve different directories in the same
context, while 'context' in a Web container's vocabulary assume 'different
application'.
- [CLONE]:
- Use a listener on the ontology to delete labels at appropriate times
(using new ontology API): this is not necessary after removing LabelIndex.
- RDF labels should not be used for private synonyms. They will screw up NE
tagging (discuss with Valy and Kali).
- [YAM] regen the parser html / add the javacc2html jar
- neither javacc2html nor javatohtml (which also covers .jj and .jjt files)
supports java 5 :-(
- [CoW] help area, URLs
- change conf/UrlMappings so that the main content area (id 2) doesn't have
the /wiki/ and id/ bits
- /action/... -> wiki 2, the main content (new)
- /wiki/action/1/... -> help (as now)
- /wiki/action/3...n/... -> other areas (as now)
- /doc/now.html -> wiki/show/2/doc/now.html
- perhaps: "/$path**?" { controller "wiki"; action "show" ... }
- the problem is that it is hard to avoid mapping /js, /css etc., and also
the g:link tags in the GSPs start behaving strangely... probably not worth
the effort!
- an edit request that misses the trailing / redirects to show; it
should redirect to edit, i.e. the page model slashURI should point to
edit instead of show when built for triage. BUT: when would such a
request be made?
- PageController bugs
- newpage flow bug: show404s on non-html/non-dir files (e.g. a
non-existent .jpg) create a new page "can't create" error
- Improved error message to suggest trying upload <IR>
- edit bug: with rich text editor can't cancel (poss only on a new page
when in w/s mode)
- can't reproduce this bug in either server or w/s mode <IR>
- SVN auth
- workstation mode shouldn't require svn authentication (why is it doing
commits? if there is a good reason then let it I guess... in which case
it does need to do authentication)
- server mode needs to do svn authentication
- infrastructure in place to associate SVN authentication information with
each wiki in the db, nicer admin interface still required.
- [CoW] PageController <HC>
- use JSecUtils to get username
- finish newpageFlow
- the exit states have to redirect conditionally depending on state
returned by npF.create, to allow proper redirecting at the end of the
flow
- use new messages template in all views/page GSPs (passing pageScope)
- put some text in new docs, from messages
- the default main index page didn't link to the help area correctly; added
a feature to backlog to be able to link between areas
- added a .yam to the default main area index
- added grails-app/i18n/cow.properties for messages and put page
controller messages there
- added a validate step in new page flow
- [CoW] misc <HC>
- added link to WikiController from Admin page, and "browse" links in the
areas list
- changed main.gsp layout to mainly mirror page.gsp; added cow favicon
- adjusted the admin URL mapping so that the Admin link doesn't get ** on
the end
- added tabindex to input pages
- better logging
- [CoW] tests <HC>
- added selenium tests
- fixed bug with non-registration of new sandboxes in WikiController
- fixed but in URL mappings that was sending /cow//plugins... requests to
PageController
- [CoW]
- upload flow <TH, IR>
- prob with jsec and uploads...
- did gwt upload... <TH>
- upgraded webtest plugin to fix issue with GWT <IR>
- auth&auth
- create a selenium test for the upload stuff
- auth&auth tests
- document the redirect to workaround the jsec/npe problems in the user
manual (hoping for a fix in grails 1.1...)
- AbstractTranslator.processUrls should use encodeAsURL. Attempted -
but java.io.URLEncoder.encode used instead. But, tests later found to fail
so commented out. actually not needed
- [CoW] security <AR>
- do we need to encode the output of the PageLinkTagLib and other tag
libraries? probably would be good to encode the query part of the output
in PageLinkTagLib. PageLinkTagLib query encoding done. All other taglibs
checked and ok.
- [YAM] dependencies <AR>
- check that quartz is still working, fix as necessary. Checked, working.
- fix Dependencies for new pages etc. Done, see svn logs for description.
- [CoW] gate.ac.uk/cow
- parser for gate.ac.uk pages <TH>
- [YAM] latex output <HC>
- fixed basic errors in tmp/yam/index.tex (cd ../../; ant yamjar);
../../bin/yam2html -llatex index.yam && tkdiff index.tex desired-index.tex
- [YAM] lists bugs <HC>
- test/resources/yam/yam-lists-bug.yam: an ordered list within a nested
unordered list which is at the current nesting level gets pushed after the
rest of the unordered list
- fixed, needed to attach to next level up in the active lists stack, not
the top-level parent
- tested using: ant -Djava.test.case=gate.yam.translate.YamTranslatorTest
yamparser java-test java-test-reports
- test/resources/yam/yam-lists-bug2.yam: the final list element gets
detached for some reason
- now attaches at the first parent list that is indented at equal or less
than
- for all this see cow/tmp in revision 957
- fixed table plus bulleted list bug (first list item always missed); added
yam-tables-bug test
- [CoW] infrastructure
- ian fixed logging work "ant java-test", e.g. from AbstractTranslatorTest;
need to run grails first and is configured via Config.groovy
- [CoW] security <AR>
- a new JsecRolePermissionRel is created even if an equal one exists. This
means that half a dozen more are added each bootstrap. need to amend
method that creates them in SecurityService.
- [CoW] security <AR> - solved by above
- Security objects do not always persist when using HSQL. For example, if
you start from an empty database, then bootstrap, an admin role will be
created. Shut down cleanly, bootstrap again, and sometimes, the admin role
has to be re-created because it was not persisted. This may be a problem
with JsecRolePermissionRels, duplicates of defaults being created many
times if you bootstrap repeatedly. Answer: check to see if this is a known
HSQL problem, perhaps fixed in a later grails, or maybe move to a
different database?
- [CoW] dependencies <AR>
- Check that pages queued for regeneration are being regenerated at some
point
- Dependency regeneration.
- If you delete database, Dependencies are not being regenerated. When
you create a new Dependencies for an existing wiki, it needs to find
all the existing dependencies between pages and record
them. Unfortunately, Dependencies is not in the default package, and
does not know about things like Wikis and YamService, and so has no
access to the regeneration code (which is in YamService). Need to move
something.
- perhaps define a dependencies service, which has a get method that
first does an exists, and if it does not, regenerates the wiki
- could also put the current YamService.regenerate in here
- [CoW] security <AR>
- JsecRolePermissionRels are being created at Bootstrap, even if one
exists. Querying with criteria, prototypes, and HQL, all fail to find
existing JsecRolePermissionRels and therefore exclude them from being
constructed again.
- [YAM] fixed width <AR, MG>
- change fixed-width text to ^...^ to reduce the clashes with equals
signs in normal text (NB changes might also be needed in
JSPWikiToYamConverter, to remove escaping of =)
- rewrite yam test files
- scripts for site conversion (MG)
- change yam documentation re fixed width and check it ok in html
- [CoW] security <AR>
- Default roles are created at bootstrap if they do not already exist. This
means that if an administrator decides legitimately to remove certain
roles or permissions, they will be recreated next bootstrap. Perhaps only
create default objects if no other objects of this type exist?
- SF 2556528. Clicking on "new page" from a user with write permission
produces and unauthorized error. Editing existing pages is
permitted. another way to test: open any html page, then change the name
in the browseraddress bar. If the current user is not admin, then an
unauthorized error comes up.
- Selenium test added for the above (previous test was for admin creating a
new page in workstation mode, which is different)
- [YAM] Dependency regeneration <AR>
- integration tests: DependenciesServicesTests
- [CoW] selenium tests <AR>
- Server user-management test suite repeaaaatedly failed at
"assertTextPresent, Edit wiki page: index.html", firefox 3.05, Linux.
Could not be replicated in the Selenium IDE, could not be replicated on a
similar linux / firefox setup.
- Duplicate: for some users, Selenium tests fail when run from ant, even
though they pass when run through the IDE, and even though wait conditions
seem ok. The tests fail in the same way if you delete .cowrc.d For
example, when Angus runs on his desktop:
- server
- IDE - succeeds when entire suite run without any breakpoints at
full speed
- ant fails at "assertTextPresent Edit wiki page: index.html" even
though there is a waitForPageToLoad and a waitForLocation just
before the step
- workstation
- IDE - succeeds when entire suite run without any breakpoints at
full speed
- ant fails in upload at "assertTextPresent Upload successful" even
though there is a waitForPageToLoad and a waitForLocation just
before the step
- [CoW] selenium tests <AR>
- Remove assumptions on user from user-writing selenium tests
- [CoW] selenium tests <AR>
- Dependency regeneration: add selenium tests for interface aspects
- regeneration button
- regeneration on updating a Wiki
- setting regeneration time
- [CoW] security <AR>
- if you try to create a new user with password "showerhead" you get a
red box with nothing in it (i.e. the message string is empty)
- TH
- simple sventon integration via /admin
- notes in the developer guide
- gettign it running without jetty (i.e. via grails builtin jetty):
- war in cow/wars
- there now exists a URL mapping to prevent /sventon being interpreted as
a directory in main
- there's a hook commented out in scripts/_Events.groovy that may or may
not be running it properly - generated 404s - so needs fixing
- HC couldn't duplicate TH's result in getting jetty to run sventon and
configuring a repo (on java 1.6.0, tried file and http urls, just got
blank pages)
- find a way to get the selenium upload test working when running in
production (e.g. via: ant -Dgate.cow.grails.command="prod run-war" grails)
- add a simple cancel button to the upload dialog
- uploading over already existing files should not work unless the user has
selected "overwrite existing files" in a checkbox in the upload dialogue
(this was present in the webflow version I think)
- it would be good if the redirect at the end of a successful upload went to
the directory view, not to index.html when it exists
- can you change the look&feel to be more like the rest of cow? e.g. use a
tooltip for the "Choose a file or ZIP archive to upload in the current
directory." message, stop using pop-up windows, instead put messages in
_messages.gsp style, don't change the spacing of the top-bar when a user
clicks upload
- [CoW, YAM] bugs
(http://sourceforge.net/tracker/?group_id=208243&atid=1004913)
- bug not on tracker/Fixed
will work with clean build? main index template doesn't exist in
production deploy (do we need production deploy now grails run-war exists?
check issue about application context path)
- bug not on tracker/Fixed
YAM regeneration doesn't work for new pages <AR,HC>
- bug not on tracker/Can't replicated
check that these old xsl bugs have been cleared up, and that it is
preserving everything except whitespace
- loss of predicate and underlining in gatewiki.html:
=svn co %(https://gatewiki.svn.sourceforge.net/svnroot/gatewiki/trunk,
https://gatewiki.svn.sourceforge.net/svnroot/gatewiki/trunk) gatewiki=
becomes =svn co
https://gatewiki.svn.sourceforge.net/svnroot/gatewiki/trunk gatewiki=
- _Set your project JDK from Platform settings._
becomes
- Set your project JDK from Platform settings.
- bug 1838819 Closed/Fixed
- create new page needs some validation and help text
- when create is pressed with nothing entered from the "new page"
directory view an error message should be displayed and the "new page"
view redisplayed
- when create is pressed with nothing entered from the "new page"
directory view with a page name containing spaces the "new page" view
should be redisplayed with a message saying page names can't contain
spaces; also, the validation should disallow other inappropriate
characters: probably just allow any letter, number, dash, dot, or
underscore
- create new page for an existing page should message "File ${new-name}
exists" and redisplay the page
- create new page shouldn't forward to edit but to show (i.e. the result
should be an empty page in show view, which can then be edited)
- validation using command object: class NewPageCommand {
// the property name has to be the same as the form field to be
// validated
String newPageName static constraints = {
newPageName(blank:false,minSize:1,matches:"[a-zA-Z0-9._-]+") } }
... def create = { NewpageCommand cmd -> ... if (cmd.hasErrors()) {
flash.message = "Invalid page name" ...
- bug 1943771 Closed/Can't replicated
- when checking for existence of a relative URI anchor parts should first
be deleted, e.g. gatewiki.html#section-3.3.
- bug 2526041 Closed/Can't replicated
- create a directory with spaces in the name (in server mode)
- then create a file there and try to edit
- svn exception due to url encoding of the spaces
- also: no reporting of the exception!
- bug 2526442 Closed/Fixed
- when in server mode, working with a wiki area that is not in
...sandboxes (e.g. /home/hamish/tmp) creating a directory and then
creating a file within it fails
- this is probably because of this code in PageService.toStaging, which
assumes that all sandboxes live in the user.home svn rep: SVNURL
pageDirSvnURL = SVNURL.fromFile(new
File("${conf.svnrep}/trunk/${pageWikiDir}"))
- bug 2537529 Open
- the shutdown-prod ant target needs to stop the jetty instance spawned by
grails
- bug (not on sourceforge)
- it would appear that you can't upload into the help wiki even if logged in
as an admin user
- I haven't tried editing pages so I don't know if that works or not
- [CoW] semantics integration
- GateService from grails-semantics-plugin <OW, DD, HC, VT>
- wraps QuestIO / OWLIM / ORDI
- translates QuestIO or Sparql queries
- send them to the ontology service
- creates XHTML and sends it back to the client
- first tasks for Olaf:
- using the GATE ontology API
- java or groovy to take an ontology URL (file: to start with) and:
- lazily initialise the GATE library (i.e. check for previous
initialisation, if none then initialise)
- check a pool of ontology LRs for one with the requested URL
- if present, return it
- else, create a GATE ontology LR for the URL
- unit tests to verify this code and to do example queries using SERQL
- a Grails service to live in the "semantics plugin"
- provide a GateController with a serql action
- this action takes a query (probably in the query part of the URL) and
returns an (X)HTML fragment representing the results
- [CoW] selenium tests
- When running selenium tests, we sometimes get:
- org.springframework.scheduling.quartz.JobMethodInvocationFailedException:
- Invocation of method 'execute' on target class [class DependenciesJob]
- failed; nested exception is java.lang.NoClassDefFoundError:
- DependenciesJob$executeclosure1
- but not when running cow normally. Maybe it is something to do with the
timing of compiling quartz jobs? or related to the "too many open files"
problem that the selenium tests often have when there is no server
running (i.e. when using "ant test-selenium"). the problem generally
seems to be when the upload test is running
- [CoW] remembered passwords
- it seems that at least in Firefox usernames and passwords are being
remembered across different instances of CoW running on different URLs.
Is this a Cow or browser bug?
- disallow changing of the path when updating wikis in WikiController (or
update the SandboxManager entries)
- add -l pretty to yam2html and test on some in-use files
- the login link appears when running in workstation mode - it shouldn't
- checked that upload works in production mode on jetty (the current SAM wiki
problem is therefore Tomcat-specific)
- when you check in CoW stuff that breaks the server start (e.g. code that
doesn't compile in Bootstrap) Hudson reports this as test failures in the
Selenium tests (which is good) but the output from those failures doesn't
show the compile errors (which is bad). is there a way to get this output? a
separate "run-server" test?
- fix deprecation warnings in gate.versioning
- get svnkit to default to LF only for all text files, inc. yam, code, etc.,
in the sandboxes
- make logfile Rolling
- why does the search form in cowpage.gsp cause the user name link in server
mode to miss the space between first and second name? fix cowpage.gsp...
- stop the cowpage.gsp layout displaying the top search box in advanced.gsp
- /admin/solr: include the advanced search form via iframe
- maybe this just gets moved to /search/advanced?
- We need eventually to have a separate index for each wiki area (i.e. for
each SVN sandbox file tree), because some users will not have permission to
search all the trees. As a simplifying assumption for the first version
we'll only support workstation mode, and search only the main wiki area.
- how to search different trees separately with Solr?
- tell solr to put it's data directory in ../nutch+solr/solr-data, not
cow/solr
- is default role given to new users, or just new registrants?
- sprint 30/03/09 to 10/04/09
- [AR] change the directory in the permission model to be a regular
expression. Then we can have directory permissions such as
"/sam(?!/internal).*"
- [AR] document directory permissions
- what empty strings mean for patterns
- how include and exclude patterns interact
- reference for regexp syntax
- what to do about slashes: the required dir will not have a leading slash
- [AR] move the svn up from cron to quartz
- if the repository is locked, skip this one
- if we get conflicts (should never happen!)... log it as an error
- provide similar facility as for the regen stuff (choose a time interval,
maybe a "do it now" button, ideally capture the update messages...)
- selenium tests
- integration tests: see comments in SvnServiceTests.groovy
- set logging appropriately
- document
- [AR, HC] versioning
- remove the "0.1" from the data directory and model it as domain class
CowDataVersion
- write generic upgrade code in Bootstrap
- [MG] convert the GATE team wiki, start running gate.ac.uk experiments
- ask HC re. create-custom-cow-site.sh script, site update protocol etc.
- [NA] search
- add directory-level security to solr
- set up indexing from the admin pages
- search via nutch+solr; admin interface for running nutch over sandboxes
(with solr indexing); user interface for searching
- /search/: accept posts from the search box on the main page view
- quartz job to collect edited/newly updated docs and index them
- support for upto 20 cores (20 sandboxes)
- separate logging for solr (as solr uses java logging)
- some documentation in gatewiki.html (move the nutch+solr.html
stuff there for developers; add some user stuff when it is working
properly)
- [HC] update create-custom-cow-site.sh so that it
Sprint ended 8th may:
- IR, MG [6, 3]
- think about the following Hudson issues:
- a job to do basic testing against the production servers and send mail
(and SMS?!) when they break
- finish sventon integration sufficient for gate.ac.uk
- move sventon config to cowrc.d/sventon-config [Done]
- configuring SVNREPs other than the default
- [Done] CoW now configures sventon with one "repository" for each wiki
area, named "wiki-N", pointing at the root of whatever repository the
top sandbox directory was checked out from (so doesn't do externals but
normal repos will work).
- base-config.properties assumes a file: URL... shouldn't?
- No longer relevant given the new config mechanism - now it just pulls
the repos root out of the svn working copy.
- HC [2 days]
- a daemon script that can be linked from /etc/init.d which does a run-prod
- [MG] versioning
- finish upgrade code
- write Bootstrap.upgrade03(List wikis) as commented
- amend gate.cow.data in Config.groovy
- tag cow and bump version to 0.3-snapshot (data version 0.3) and upgrade
the various installations
- document tables / lists newlines thing
- NA, HC [NA 5, AR 0.2 days]
- /search/advanced: maybe do the same as for advanced? [Done]
- removed this page and added other indexing related advanced features on
the administerIndex.gsp page. There isn't much on the advanced search
page anyway. Almost everything there is already used in our search
functionality.
- a page to list what is in the indexing queue [Done]
- Implement a queue for indexing of wiki areas [Done]
- move crawl-urlfilter.txt to urlfilter-default.txt with only static content
in it. create crawl-urlfilter.txt at runtime for filtering. [Done]
- installation loose ends arising from the sprint
- sam repository: turn off cron and use quartz
- team wiki: deploy and go live
- gate.ac.uk: think further if it makes sense on gatewiki
- gate-top index and install on gate-compute2 [Done]
- 0.3
- upgrade sam wiki [MG, IR]
- tag the gatewiki repository as 0.3 [MG]
- fix dependencies - dependent files are not being committed after changes
[MG]
- a daemon script that can be linked from /etc/init.d which does a run-prod
[MG]
- gate.ac.uk trialing [IR, MG, HC]
- Hudson testing against the production instances - heartbeat runs on most
servers[MG]
- document tables / lists newlines thing [HC]
- breadcrumbs: see http://www.useit.com/alertbox/breadcrumbs.html; a simple
design where the directories above the current page are listed along with
the top-level is probably enough, e.g. for the page doc/yam/summary.html we
would have "You're here: Home > Help > Doc > Yam" as breadcrumbs. see also
Darryl Pentz discussion in Jan on the mailing list. [NA]
- changed the behaviour of PageController.show and HtmlTranslator so:
- /page/show/... is never thought to be an non-existant page during
translation (so is not coloured red)
- non-existant pages have a m[issing]=1 parameter added to the url
- only non-existant pages with the missing param are forwarded to create new
page
- i.e. it will now only offer to create missing pages when they were found
to be non-existant within the current area during translation.
- evaluated the grails nav plugin:
- the Navigation Plugin gives you site navigation menus by convention. You
just define a property in your controllers to determine what actions are
shown and in what groups. http://www.grails.org/Navigation+Plugin
- this is nice, but from the mailing list it seems not without issues, and
we would have to do some work to figure out how to add it into layouts,
how to have several instances for side bar and so on, and also how to
populate it from YAM lists, which are the obvious basis for navigation in
CoW. so: I guess we won't use this for now
- implemented Navigation for Specific Directory Trees:
- in PC, where it said pageBody = FileUtils.getBody(pageFile), changed
this to also do PageService.getNavigationModel(pageFileDir, wiki.path)
- for each layout file in conf.navigation.files
- find the nearest one to current dir, up the tree to wiki top
- get its body
- set as var in the model to pass to the layout
- layouts for whole wiki-areas [NA]
- add a "areaLayoutName" to Wiki, which would reference a layout GSP in the
site-specific plugin; PageLinkTabLib would then use this for the layout in
that wiki area
- modify PageLinkTaglib.layoutLink
- the check to see which area layouts are present should be done in
Bootstrap in fact, and store the list of them in a config var, just like
setGuestLayoutPath() does when called from Bootstrap now
- non-native pages like javadocs: [NA, HC]
- any directory that now contains a file called .cow:raw-html will now
render any non-YAM HTML files directly (see
gatewiki.html#raw-html). this means you can serve e.g. javadocs.
BUT it is a potentially nasty security hole and is turned off by
default, so we need to add a better mechanism:
- add a list of path patterns to each wiki area, which would match either
files or directories (AR might know the GORM invocations? a new table
RawHtmlPatterns?)
- do an Ant API path match in PageController.show, according to config
settings (IR would know the Ant API call?)
- when a non-native HTML (see the developer
guide discussion path matches serve it raw as with the .cow:raw-html
mechanism
- check this works for javadocs and for other gate.ac.uk pages (tao etc.)
- do we need a link back to the Wiki area from pages served raw? this
could be nice, but a) isn't typical, e.g. for javadocs, and b) would
need some munging of the page, e.g. by insertion (maybe of iframe?) just
after the body tag or similar. so perhaps not worth it? [didn't do this]
- finish sventon integration [IR]
- proper proxying of sventon requests through a CoW controller
(/admin/sventon)
- sventon pages contain all sorts of absolute path links: <a
href="/sventon/repos/...">, <script src="/sventon/js/sventon.js" />,
<div id="/sventon/ajax/..."> (where /sventon is the context path where
sventon is deployed)
- need to fix these up to point through cow, i.e. /sventon/repos ->
/admin/sventon/repos (for root-level CoW) or /sam/sventon/ajax ->
/sam/admin/sventon/ajax (for Cow deployed under /sam)
- sventon pages include a <base> element, need to fix that up too. This
will break any relative links in the CoW top banner (if there are any).
- /ajax and /js need to be proxied raw, /anythingelse needs a CoW banner
- thankfully, sventon.js uses a global JavaScript variable for the
contextPath, so don't need to massage the JavaScript code (phew).
- when fetching the proxied content from sventon, add
;jsessionid=<session-id-from-cow> to the URLs, to keep sventon and cow
sessions in sync. Any jsessionid in URLs generated by sventon need to
be stripped during the HTML massaging process.
- once proxying works, we can link from pages direct to their history.
- simpler idea:
- remove the localhost filter, allowing direct access to /sventon, but...
- add a filter that checks for an object in the session before permitting
access.
- this object is put in the session by CoW, posting to a URL that is
still restricted to localhost only.
- grails 1.1.1 [IR]
- MG recommendation: The one thing I would change is to set the DTD to HTML
4.01 and not XHTML. The file isn't valid XHTML and the server is pushing the
file out as text/html so it makes no sense to use the XHTML DTD. Also strict
browsers will start to behave strangely if the XHTML isn't valid and may not
display anything at all.
- conflicts no longer being reported properly in edit flow
- fixed - in the case of repository changes to a file, PageService was using
the repository version of the file in its entirety, rather than doing any
merging.
- when you do an edit and the svn auth fails, there is an exception thrown
(org.tmatesoft.svn.core.SVNException: svn: CowAuthenticationManager only
provides a single authentication token) but it isn't reported to the user
- fixed - any exceptions that occur in the edit flow (and are picked up by an
on(Exception).to "errorExit") are reported to the user on the redirected
show page. SVN auth errors present a custom error message, other exceptions
just print a generic message, but special cases for other exception types
are easy to add.
- jape yamalyser
- fix the jasper [IR, HC]
- fixed - I've repackaged GATE's copy of jasper-compiler-jdt.jar into a
different package name using JarJar
links so it no longer conflicts with the (slightly older) version
provided in grails/lib.
- spring xgapp pool [IR]
- Add a text box for svn log message (in server mode).
- Added plugins facility to YAM and did Google and Twitter plugins.
- Renamed web-app/images to web-app/__images. Documented remaining top-level
namespace conflicts in main area wiki.
- Implemented NA's solution for raw HTML in iFrame; managed to reuse the show
action to do the heavy lifting, and added a print facility.
- Turned off the dev mode logo to make debugging easier.
- Generalised PageLinkTagLib.createPageURIBase(attrs) to allow for multiple
query elements (e.g. ?type=dir&print=1)
- Failed miserably to write any tests for the new stuff.
- Implemented a single "Edit" button and allow user prefs re. what type of
editor for yam htmls (default to form-based)
- If there is a logged in user, store preference per user in the DB
- If there is no logged in user (workstation mode or pages with anon write
permission), store preference in a browser cookie.
- Sventon history view now uses an iframe surrounded by the usual CoW header,
breadcrumb trail and a link back to "show" the page whose history you are
viewing.
- issue about what to do with unsuccessful edits
- edit cancellations now svn revert and delete the staging copy, so the next
edit will start from clean.
- doing some tests for the FckEditor (should be ok now we're on selenium not
canoo?). there was a bug report that said all the list nesting disappeared
after an edit - might have been operator error of some sort, but would be
good to check. also might be good to upgrade to latest FckEditor if there's
a newer release
- The loss of nesting is due to the fact that the HTML-to-yam XSL gets
confused by the fact that we now nest the nested <ul> inside the parent
<li>, and it adds a blank (well, whitespace-only) line before the first
item of the nested list. This is now fixed in HtmlToYamConverter.
- when creating a user with a bad password field I got this error:
- Password must contain at least two digits
- Server Error :-(
- Error Details
- Message: Cannot invoke method hasProperty() on null object Caused by:
Error processing GroovyPageView: Error executing tag <g:form>:
java.lang.NullPointerException: Cannot invoke method hasProperty() on null
object
- Fixed by passing the actual UserCommand object in the model for the render
call, rather than passing a map.
- jape yamalyser [HC, NA]
- do wrapping
- recognise inter-unit gaps and create Edit annotations with "numBlankLines"
attr
- do edits
- Solr search issues [NA]
- when you say "index" from the admin pages, probably it should delete the
previous index? or at least offer you the option
- when you fail to delete the previous index before re-indexing, results
from outside of the current sandbox (e.g. you previously indexed some
other sandbox then moved to a new one) will be presented as file:// with
the full path; this is a security hole and should never happen
- searching for a quoted string e.g. "software architecture for language
engineering" doesn't work; is this supported by the soly QL? perhaps put a
summary of that QL at the bottom of the search results page?
- fix run-prod [IR]
- ant run-prod was giving NPE in eventConfigureJetty - fixed locally,
GRAILS-5055 in JIRA - now getting a different NPE (GRAILS-4673) which will
be fixed when we upgrade to grails 1.2. In the meantime we work around
this by treating DataSource.groovy as an external configuration file (via
grails.config.locations) loaded from the config directory, rather than
as a standard compiled class in grails-app/conf.
- find a better way to avoid showing the search box in the admin area than
checking for null id
- Now conditions on webRequest.controllerName to only show the box for
PageController and SearchController requests.
- ehcache: "jsecurity seems to create a dir called /tmp/jsecurity-ehcache/.
prob is that when you run several cows as different people it creates perms
probs" possible solution: http://www.jsecurity.org/node/1046
- Now generates ehcache config at BootStrap time pointing to a temp dir
under cowrc.d
- Layout issues
- improve the float layout, e.g. to deal with top-bar links scrolling to
under the logo (try yui frames thing from GIA book?)
- ditch the jquery menus and have static html there.
- GAU
- g8rs.net
- run fail2ban
- ge1/2 cross-snapshots (inc. /etc and /home) into /home/backup
- stop running on 8080
- externals in gau-main-tree for sale, gate etc.
- browser compatibility testing [IR]
- start pushing TAO into the g8rs main wiki area [IR]
- add a secure rsync from hudson to push the TAO build (minus .svns) into
ge1:/data/herd/g8rs-data/gau-main-tree/sale/tao (make sure that there's a
.cow:no-iframe file in the directory that Tex4HT creates)
- ref in the tex4ht css in the gau cowguest.gsp.
- allow ordering of search results by recency [NA]
- move .grails to cow/dot-grails to avoid conflicts on server start
- dependencies issue [IR]
- the serialised dependencies files should include the run mode in the name
(dev, prod, test), or else they may be saved relative to one wiki in e.g.
dev mode then restored relative to another in production mode?
- Sventon has a line of links that let you look up the tree to parent
directories. Changed so that only those levels that the current session has
permission on will be clickable [IR]
- turned off regen of sandboxes from bootstrap (make it a config option) [HC]
- removed the border style from YAM table elements
- start pushing TAO and javadoc into the g8rs main wiki area [IR]
- make sale/tao, javadoc dirs and download/snapshots on g8rs non-writeable
(so people don't try and edit non-svn content there) [note: I've added a
topdoc/systems-admin section under GE1 stuff for rules like this that we
need to remember for the migration]
- possible issues to check up on [HC, IR]
- will it manage to serve GB movies?! otherwise some cludge with apache
serving gate-big? Just tried the big demo movie and it seemed to work for
me. Not sure how it'll cope under load though. (IR)
- am I right to remember that when checking out to staging it first does a
depth 0 co then an up on the edited file? if not we may end up with very
big staging areas? That is correct (IR)
- edit of /credits.html (which is a raw HTML file) doesn't work (some markup
problem seems to prevent the execution of the FCK javascript) [IR]
- fixed the FckEditorTagLib to use encodeAsJavaScript on all values that go
into JS string literals, which has fixed this.
- solr [NA]
- try searching on http://gate.ac.uk/... too many open files bug
- when matching the path prefix on the file:// uri for search result hits,
if the prefix doesn't match then log an error and throw away that hit (the
error text: "failed to match hit prefix; this probably means the sandbox
was moved on disk and the index needs regenerating")
- please add a field that indexes file names (e.g. business.html)
- upload [IR]
- upload of .tgz fails without message
- upload of .zip of a directory claims to work but creates a zero-length
file for directories
- frame the mailing list archives (sf, gmane, ...) on
new-trunk/gau-main-tree/mail/archives.html (not a yam page, just html) [NA]
- user guide to logging in, editing etc., revise gatewiki.html [HC]
- g8rs.net [HC, IR]
- CSS the twitter div a little to make it different (diff colour? make the
URLs within it smaller to hide their ugliness?) [IR]
- contact page [NA]
- gate.cow.ContactController with a single default action which
- displays a captcha (see RegisterController and also Config.groovy for
the captcha setup) and allows sending of an email to gate.cow.
- tell grails to also run on https and get jetty to redirect logins there
(done, though we'll not use this in production) [IR/MG]
- buy a ssl certificate for *.gate.ac.uk [HC]
- secure login
- 443 handling via apache VH [IR]
- Infrastructure done, using a temporary certificate until we get the real
one.
- analyse the security holes found by ratproxy [MG]
- javascript in gau site plugin to replace bitly links in the twitter with
"Read more..." [MG]
- delete flow [IR]
- directory-level view
- change to 3-column table
- file type icon
- file name
- select checkbox
- make a page containing a table with two rows, one of which is empty. Save or
preview results in the entire page content being dropped without warning.
(i.e. the error reporting seems broken; faults in table syntax generate yam
errors; need to look into this)
- The underlying parser error for this has now been fixed but the principle
still stands that we should handle yam parsing errors more cleanly. [IR]
- in the edit flow, the log message is not stored in the flow under some
circumstances (probably from preview to continue editing) [IR]
- I get a fair amount of flicker between menu items; is it possible to get the
selection mechanism to work a little slower or something? (fixed, IR)
- the iframe/frame combination on the /mail page is messy... let's just link
to the pages and delete the frame stuff [HC]
- lose all the XML-style tag closing stuff in YAM output, we're going with
HTML not XMTML [HC]
- get the hudson jobs to also do doc and to push the results (and the test
results - see the links from cow/doc/index.html) [IR]
- titles including markup incorrectly include the markup in the html title
(e.g. %br puts a <br/> into the title) [MG]
- when editing a page with no title the leading blank lines get removed [MG]
- when previewing an edit the links are wrong (they point to "/page/edit..."
locations); can we use a BASE directive to solve this? [MG]
- change "switch to rich editor" to "switch to WSYWIG editor" [MG]
- an empty .html causes NPE when served [MG]
- finish the documentation on Firefox profiles, specifically how you create
them to remember certificates [MG]
- get edits of raw html to also include the headers [HC]
- make use of recaptcha an option, and get audio to work this way (user
credentials to go into .cowrc.groovy on ge1) [MG]