getting around slow publishing jobs

Favorite-off Notification-off
43 Thumb-up Thumb-down
Idea by earl.fogel about 1 year ago Favorite-off - Open

It would be nice if we could pause a big publish job to let other smaller jobs through, and then resume it later.

Alternatively, we’d like to be able to set a “width” on the publish queue, or have multiple queues, so more than one job could publish at a time.

18 comments

  • 1 point Thumb-up Thumb-down Favorite-off by slota.benoit about 1 year ago Permalink
    Otherwise it would be nice to set priority of publishing (for users, groups, target, etc...)
  • 0 points Thumb-up Thumb-down Favorite-off by irifkin about 1 year ago Permalink
    Publishing can be quite slow and I can understand the desire to pause particularly large jobs. Why does it need to be a serial process? Question is more for HH than earl.fogel

    I suppose priority could have its uses, but who would set priority? Wouldn't everyone think there publish job is top priority? I'm not sure how useful it would be for us, but it's an interesting idea.
  • 3 points Thumb-up Thumb-down Favorite-off by kevin about 1 year ago Permalink
    The publisher process definitely needs to be overhauled to allow multiple pages to be published simultaneously.  It is painful waiting for a list of 20+ pages/assets to publish.  While reordering within the publisher queue is definitely a nice feature, it wouldn't be necessary if the publishing queue was faster.  Just my two cents.
  • 1 point Thumb-up Thumb-down Favorite-off by wabaus 11 months ago Permalink
    In v.6.x with Sites, could idle load-balanced servers *each* look down the queue for a job that is not in the same "Site" as the currently active job(s)? I think the issue with the queue being linear and not load-balanced is avoiding race conditions where 2 jobs affect the same file, and the earlier big job might over-write a later small job that completed first. Flagging the "Site" for each job might be enough protection against this to at least keep our Law School from slowing down our Business School's publish jobs...
  • 2 points Thumb-up Thumb-down Favorite-off by alharmon 5 months ago Permalink
    With the introduction of sites in 6.x site managers experience all of the look and feel of operating w/in their own semi-virtual instance of CS until it comes to publishing -and then things screech to a grinding halt.  The benefits of sites w/in 6.x quickly hit diminishing returns and whither in the shadow of the poor publishing model.  A faster publishing queue is *imperative with either simultaneous publishing or reordering of the publishing queue. 
  • 1 point Thumb-up Thumb-down Favorite-off by wabaus 5 months ago Permalink
    I totally agree that multiple queues/jobs (even if just one job per site) would be a huge gain.

    Re: reordering - Are you familiar with the role-level permission you can grant to any role you trust with the power.  It's under "Home Area Abilities: Publishing" look for "Reorder the publish queue: Enabled".
  • 4 points Thumb-up Thumb-down Favorite-off by jpappas 2 days ago Permalink
    I vote for multiple queues. It's silly to do anything else.
  • 1 point Thumb-up Thumb-down Favorite-off by nfewell 1 day ago Permalink
    I vote for multiple queues.
  • 1 point Thumb-up Thumb-down Favorite-off by jraller 1 day ago Permalink

    Even moving up to two queues where large jobs go into the large queue and single pages go into the small queue would be an improvement because it would automatically sort the single page jobs so they could go through faster.

    I’d also like to see Hannon Hill offer a separate publishing server. That would allow us to separate out publishing from editing so that when Queue times got large you could just license and add another publishing server(or better yet, not even have to license them). A small install would still have a Cascade Server and a database server, but a larger one could have several publish servers as well. If the publish servers are part of a virtual environment their resources wouldn’t be sitting idle between publish jobs as they could be shifted to other tasks.

  • 0 points Thumb-up Thumb-down Favorite-off by tgatkins 1 day ago Permalink
    When implemented at an enterprise level across a large institution, multiple publishing queues is essential. This should include at least one "priority" queue, available for use to only a small group of specific users, reserved solely for mission-critical jobs.
  • 0 points Thumb-up Thumb-down Favorite-off by TonyDunn 1 day ago Permalink
    I agree with tgatkins. It would be nice as well, to tie publishing sets to specific queues, so that priority jobs could automatically go to the top of the queue when triggered by a workflow (for example).
  • 0 points Thumb-up Thumb-down Favorite-off by cbarry 1 day ago Permalink
    I vote for multiple queues.
  • 0 points Thumb-up Thumb-down Favorite-off by jkreuzig 1 day ago Permalink
    Publishing has been an issue for us as well.  I think the idea of having a site-specific publishing queue may work.  Of course, once you have assets linked across sites you end up with some of the same issues (asset locking, race conditions, etc.) as the current publishing model.

    I'm thinking out loud here, so YMMV.  If there was a way to "snapshot" all components used in a publishable asset when it's placed in a queue, you should be able to have multiple queues.
  • 0 points Thumb-up Thumb-down Favorite-off by amcmilli 1 day ago Permalink
    I also vote for multiple queues.  We get very few complaints from our user community about Cascade but waiting on publishing is one thing that they have complained about.
  • 0 points Thumb-up Thumb-down Favorite-off by wabaus 1 day ago Permalink
    As I recall, there is a "snapshot" effect applied to an asset when it is published.  Not sure how far down the dependency chain it goes.  It could be complete, as Cascade knows the timestamp of the publish job – but I don't know if it is.

    Re-ordering the queue is "at your own risk" and could in theory cause an older version to be published on top of a newer version.  (How "smart" is "smart publishing" in noticing/mitigating race conditions?)  Maybe I'll test this and see...

    But if it is a problem, Priority Queues and fully Parallel Queues might be tricky.  While I'd be all for that approach... If it would be easier (i.e., we'd see it in Cascade sooner) I'd settle for a per-site parallel queue for jobs isolated to different sites.  This would at least keep the Law and Business schools from slowing each other down.
  • 0 points Thumb-up Thumb-down Favorite-off by wabaus 1 day ago Permalink
    I posted a request on another tangent for speeding up publishing of small jobs here:
  • 0 points Thumb-up Thumb-down Favorite-off by kevin 1 day ago Permalink
    Multiple queue's would not be necessary if the publisher published multiple files simultaneously.  Why now allow 10 items to publish at the same time? If you are concerned about the same file being in the publish queue more than once, keep track of the time entered and force those to publish in order, otherwise publish simultaneously. If you want to place more weight on certain folders/file that is fine. 
  • 0 points Thumb-up Thumb-down Favorite-off by jkreuzig 1 day ago Permalink
    "How "smart" is "smart publishing" in noticing/mitigating race conditions?"

    Andrew,

    I sure I will be corrected on this if I'm wrong, but I believe "smart publishing" only has an effect on file assets.  File assets (such as images) are not managed the same way in cascade, so cascade checks to see if the asset on the server being published to is older that the version in cascade.  If not, it skips publishing it.  Any fully-managed asset in the cms (page, block, etc.) needs to be rendered every time.  So if you have an image heavy site that does not have many changes on it, it will publish slowly the first time and much faster on subsequent publishes.

    I do like the idea of per-site queues.  While it may take more resources (memory & CPU), that's a trade off I'm willing to make.

No attachments


* Note - To input code samples, click the pencil icon (this will remove the WYSIWYG) then be sure to start and end code sections with @@@ (three '@' signs).  For more information on textile markup, click here.  

Sign in

Follow us on Twitter

Current Category