From a65fb48903acf790fb208e467f3cb53442f1ef86 Mon Sep 17 00:00:00 2001 From: Justin Dalrymple Date: Sun, 24 Sep 2017 09:22:37 -0400 Subject: [PATCH] Adding pagination support --- README.md | 48 ++++++++++++++++++++++++++-------------- docs/project-members.md | 20 +++++++++++------ docs/project-triggers.md | 16 +++++++++----- docs/projects.md | 24 ++++++++++++-------- src/Models/BaseModel.js | 5 +++-- 5 files changed, 74 insertions(+), 39 deletions(-) diff --git a/README.md b/README.md index a9bd7e3a..96145677 100644 --- a/README.md +++ b/README.md @@ -2,14 +2,13 @@ [![NPM](https://nodei.co/npm/node-gitlab-api.png?downloads=true&stars=true)](https://nodei.co/npm/node-gitlab-api/) -node-gitlab-api -=============== +# node-gitlab-api [GitLab](https://github.com/gitlabhq/gitlabhq) API Nodejs library. It wraps the HTTP v4 api library described [here](https://github.com/gitlabhq/gitlabhq/tree/master/doc/api). -Table of Contents -================== +## Table of Contents + * [Install](#install) * [Usage](#usage) * [Docs](#docs) @@ -18,16 +17,14 @@ Table of Contents * [License](#licence) * [Changelog](#changelog) -Install -======= +## Install ```bash # Install from npm npm install node-gitlab-api ``` -Usage -===== +## Usage URL to your GitLab instance should not include `/api/v4` path. @@ -67,26 +64,45 @@ GitlabAPI.projects.create(projectId, { }) ``` -Docs -==== +### Pagination + +For any .all() function on a reasource, it will return all the items from gitlab. This can be troublesome if there are many items, as the request it self can take a while to be fulfilled. As such, a maxPages option can be passed to limit the scope of the all function. + + +```javascript +// Listing projects +let projects = await gitlab.projects.all({max_pages:2}); + +``` + +You can also use this in conjunction to the perPage argument which would override the default of 30 per page set by Gitlab: + +```javascript +// Listing projects +let projects = await gitlab.projects.all({max_pages:2, per_page:40}); + +``` + + +## Docs + Although there are the offical docs for the API, i realised i should still explain the function calls in this library, so i wrote some up! * [Projects](https://github.com/jdalrymple/node-gitlab-api/blob/master/docs/projects.md) -Contributors -============ +## Contributors + This started off as a fork from [node-gitlab](https://github.com/node-gitlab/node-gitlab) but I ended up rewriting 90% of the code. Here are the original work's [contributers](https://github.com/node-gitlab/node-gitlab#contributors). - [Dylan DesRosier](https://github.com/ddesrosier) -License -======= +## License MIT -Changelog -========= +## Changelog + [1.0.14](https://github.com/jdalrymple/node-gitlab-api/b8fb74828503f0a6432376ad156b7f9e33f6228e) (2017-08-1) ------------------ - Adding default file name for file uploads. If none is supplied, the filename is diff --git a/docs/project-members.md b/docs/project-members.md index ed43f0fe..b979bb9d 100644 --- a/docs/project-members.md +++ b/docs/project-members.md @@ -17,7 +17,7 @@ GitlabAPI.projects.share(projectId, groupId, groupAccess, { // params }); ``` -Parameters: [Share a project with a group](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#share-project-with-group) +**Parameters**: [Share a project with a group](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#share-project-with-group) ### Delete a shared project link within a group @@ -26,7 +26,7 @@ Unshare the project from the group. ```javascript GitlabAPI.projects.unshare(projectId, groupId); ``` -Parameters: [Unshare a project with a group](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#delete-a-shared-project-within-group) +**Parameters**: [Unshare a project with a group](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#delete-a-shared-project-within-group) ### List all members @@ -35,7 +35,13 @@ Gets a list of project members viewable by the authenticated user. ```javascript let members = GitlabAPI.projects.members.list(projectId); ``` -Parameters: [List all members](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#list-all-members-of-a-group-or-project) +**Parameters**: [List all members](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#list-all-members-of-a-group-or-project) + +**Extra Parameters** + +| Argument | Description | Type | Required | Default | +|---------------|--------------------------|----------|----------|-------------------| +| max_pages |Limits the amount of pages returned | Number | No | All pages | ### Get a member @@ -44,7 +50,7 @@ Gets a member of a project. ```javascript let member = GitlabAPI.projects.members.show(projectId, memberId); ``` -Parameters: [Get a member](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#get-a-member-of-a-group-or-project) +**Parameters**: [Get a member](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#get-a-member-of-a-group-or-project) ### Add a member @@ -55,7 +61,7 @@ let member = GitlabAPI.projects.members.add(projectId, { // params }); ``` -Parameters: [Add a member](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#add-a-member-to-a-group-or-project) +**Parameters**: [Add a member](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#add-a-member-to-a-group-or-project) ### Edit a member @@ -66,7 +72,7 @@ let member = GitlabAPI.projects.members.edit(projectId, { // params }); ``` -Parameters: [Add a member](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#add-a-member-to-a-group-or-project) +**Parameters**: [Add a member](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#add-a-member-to-a-group-or-project) ### Remove a member @@ -75,4 +81,4 @@ Removes a member of a project. ```javascript GitlabAPI.projects.members.remove(projectId, memberId); ``` -Parameters: [Remove a member](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#remove-a-member-to-a-group-or-project) +**Parameters**: [Remove a member](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/members.md#remove-a-member-to-a-group-or-project) diff --git a/docs/project-triggers.md b/docs/project-triggers.md index ddc5a3f3..4cb4d067 100644 --- a/docs/project-triggers.md +++ b/docs/project-triggers.md @@ -14,7 +14,13 @@ Allow to share project with group. // From a project ID let triggers = GitlabAPI.projects.triggers.list(projectId); ``` -Parameters: [List all project triggers](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#list-project-triggers) +**Parameters**: [List all project triggers](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#list-project-triggers) + +**Extra Parameters** + +| Argument | Description | Type | Required | Default | +|---------------|--------------------------|----------|----------|-------------------| +| max_pages |Limits the amount of pages returned | Number | No | All pages | ### Get a trigger @@ -24,7 +30,7 @@ Get details of project's build trigger. // From a project ID let trigger = GitlabAPI.projects.triggers.show(projectId, triggerId); ``` -Parameters: [Get trigger details](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#get-trigger-details) +**Parameters**: [Get trigger details](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#get-trigger-details) ### Add a trigger @@ -36,7 +42,7 @@ let trigger = GitlabAPI.projects.triggers.add(projectId, { // params }); ``` -Parameters: [Create a trigger](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#create-a-project-trigger) +**Parameters**: [Create a trigger](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#create-a-project-trigger) ### Edit a trigger @@ -48,7 +54,7 @@ let trigger = GitlabAPI.projects.triggers.edit(projectId, triggerId, { // params }); ``` -Parameters: [Edit a trigger](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#update-a-project-trigger) +**Parameters**: [Edit a trigger](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#update-a-project-trigger) ### Remove a trigger @@ -58,4 +64,4 @@ Remove a trigger from a project. // From a project ID let trigger = GitlabAPI.projects.triggers.remove(projectId, triggerId); ``` -Parameters: [Remove a trigger](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#remove-a-project-trigger) +**Parameters**: [Remove a trigger](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/pipeline_triggers.md#remove-a-project-trigger) diff --git a/docs/projects.md b/docs/projects.md index 8842ec04..00a45dde 100644 --- a/docs/projects.md +++ b/docs/projects.md @@ -23,7 +23,13 @@ Get a list of visible projects for authenticated user. When accessed without aut let projects = GitlabAPI.projects.all(); ``` -Parameters: [List all projects](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#list-projects) +**Parameters**: [List all projects](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#list-projects) + +**Extra Parameters** + +| Argument | Description | Type | Required | Default | +|---------------|--------------------------|----------|----------|-------------------| +| max_pages |Limits the amount of pages returned | Number | No | All pages | ### Get a single project @@ -39,7 +45,7 @@ let projectA = GitlabAPI.projects.show(21); let projectB = GitlabAPI.projects.show('diaspora/diaspora'); ``` -Parameters: [Get a single project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#get-single-project) +**Parameters**: [Get a single project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#get-single-project) ### Create a project @@ -51,7 +57,7 @@ let projectA = GitlabAPI.projects.create({ // params }); ``` -Parameters: [Create a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#create-project) +**Parameters**: [Create a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#create-project) ### Create a project for user @@ -64,7 +70,7 @@ let projectA = GitlabAPI.projects.create({ // params }); ``` -Parameters: [Create a project for user](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#create-project-for-user) +**Parameters**: [Create a project for user](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#create-project-for-user) ### Edit a project @@ -76,7 +82,7 @@ let projectA = GitlabAPI.projects.edit(projectId, { // params }); ``` -Parameters: [Edit a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#edit-project) +**Parameters**: [Edit a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#edit-project) ### Fork a project @@ -88,7 +94,7 @@ let projectA = GitlabAPI.projects.fork(projectId, { // params }); ``` -Parameters: [Fork a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#fork-project) +**Parameters**: [Fork a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#fork-project) ### Star a project @@ -98,7 +104,7 @@ Stars a given project. Returns status code `304` if the project is already starr ```javascript let projectA = GitlabAPI.projects.star(projectId); ``` -Parameters: [Star a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#star-a-project) +**Parameters**: [Star a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#star-a-project) ### Unstar a project @@ -108,7 +114,7 @@ Unstars a given project. Returns status code `304` if the project is not starred ```javascript let projectA = GitlabAPI.projects.unstar(projectId); ``` -Parameters: [Unstar a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#unstar-a-project) +**Parameters**: [Unstar a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#unstar-a-project) ### Remove a project @@ -118,4 +124,4 @@ Removes a project including all associated resources (issues, merge requests etc ```javascript GitlabAPI.projects.remove(projectId); ``` -Parameters: [Remove a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#remove-project) +**Parameters**: [Remove a project](https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/projects.md#remove-project) diff --git a/src/Models/BaseModel.js b/src/Models/BaseModel.js index b75efd59..f6c054cd 100644 --- a/src/Models/BaseModel.js +++ b/src/Models/BaseModel.js @@ -3,10 +3,11 @@ const LinkParser = require('parse-link-header'); async function getAllPages(client, endpoint, options, results = []) { const response = await client.get(endpoint, options, true); const links = LinkParser(response.headers.link); + const limit = options.max_pages ? response.headers['X-Page'] <= options.max_pages : true; const moreResults = results.concat(response.body); - if (links.next) { + if (links.next && limit) { await getAllPages(client, links.next.url.replace(client.url, ''), options, moreResults); } @@ -19,7 +20,7 @@ class BaseModel { } get(endpoint, options) { - if (!options.page && !options.per_page) { + if (!options.page) { return getAllPages(this.client, endpoint, options); }