Need to load data from a file

How Do I...
#1

the script.js uses and array that is generated.
function shuffle(str) {
return str
.split(’’)
.sort(function() {
return 0.5 - Math.random();
})
.join(’’);
}

// For demonstration purposes we first make
// a huge array of demo data (20 000 items)
// HEADS UP; for the _.map function i use underscore (actually lo-dash) here
function mockData() {
return .map(.range(1, 20000), function(i) {
return {
id: i,
text: shuffle(‘te ststr ing to shuffle’) + ’ ’ + i,
};
});
}
(function() {
// init select 2
$(’#test’).select2({
data: mockData(),

I would like to read the data from a file

#2

here is my attempt… This doesn’t work
function getDevs() {
var fs = require(‘fs’);
var DEVS = fs.readFileSync(‘data/Device_Hash.txt’).toString().split("\n");
return DEVS;
}
(function() {
// init select 2
$(’#test’).select2({
data: getDevs(),

#3

Select2 requires its data in a specific format. Does your getDevs() function return a JavaScript object in that exact format? If not, you need to make it do so. (My guess is that since you’re reading the data from a text file, it is not in the required format.)

#5

sorry didn’t see the link. Ok I have my data formatted in the correct json payload.

example:
{
“results”: [
{
“id”: 1,
“text”: “aandca-ss-5-a-up”
},
{
“id”: 2,
“text”: “aandca-ss-5-b-up”
},
{
“id”: 3,
“text”: “aandca-ss-6-a-up”
},
{
“id”: 4,
“text”: “aandca-ss-6-b-up”
},
{
“id”: 5,
“text”: “aandca-ss-7-a-up”
},
{
“id”: 6,
“text”: “aandca-ss-7-b-up”
},
…
{
“id”: 21956,
“text”: “wapcbb-singapore0-att-pe”
}
],
“pagination”: {
“more”: true
}
}

However, It doesn’t produce any data in the dropdown.

here is my script.js:
function getDevs() {
var fs = require(‘fs’);
var DEVS = fs.readFileSync(‘data/devices.json’).toString().split("\n");
return DEVS;
}
(function() {
// init select 2
$(’#test’).select2({
data: getDevs(),
placeholder: ‘search’,
multiple: true,
// query with pagination
query: function(q) {
var pageSize,
results,
that = this;
pageSize = 30; // or whatever pagesize
results = [];
if (q.term && q.term !== ‘’) {
results = _.filter(that.data, function(e) {
return e.text.toUpperCase().indexOf(q.term.toUpperCase()) >= 0;
});
} else if (q.term === ‘’) {
results = that.data;
}
q.callback({
results: results.slice((q.page - 1) * pageSize, q.page * pageSize),
more: results.length >= q.page * pageSize,
});
},
});
})();

here is the index.htnl:

Select Devices for Maintenance

What am I doing wrong?

#7

I still think your getDevs function is returning an array, not an object (because it’s split()ting a string value, which will return an array of strings). Instead, getDevs should call JSON.parse() on the data from the file, which will return the object Select2 expects.

#8

Hi John, Thanks for the quick response, I do appreciate your help. I am sure you have realized by now that I am very new at this. Could you show me an example on how to call the JSON.parse?

Thanks

#11

I have succeeded in loading my json data. I do see results in chrome (F12) for all my devices in my list. However, the dropdown does not populate when I do a search.

Here is my latest code:
$("#getDevs").select2({
ajax: {
url: ‘data/devices.json’,
dataType: ‘json’,
delay: 750,
data: function (params) {
return {
q: params.term, // search term
page: params.page
};
},
processResults: function (data, params) {
// parse the results into the format expected by Select2
// since we are using custom formatting functions we do not need to
// alter the remote JSON data, except to indicate that infinite
// scrolling can be used
params.page = params.page || 1;

return {
results: data,
pagination: {
more: (params.page * 30) < data.total_count
}
};
},
cache: true
}

})

when I click in the text area I get this in results:

  1. {results: [{id: 1, text: “aandca-ss-5-a-up”}, {id: 2, text: “aandca-ss-5-b-up”},…],…}

  2. pagination: {more: true}

  3. more: true

  4. results: [{id: 1, text: “aandca-ss-5-a-up”}, {id: 2, text: “aandca-ss-5-b-up”},…]

  5. [0 … 9999]

  6. [10000 … 19999]

  7. [20000 … 21955]

I have 22K devices and when I click on the id list I do see all of my devices. But when I click on the text area or search nothing appears in the dropdown box

#13

You cannot specify the data property in your Select2 initializer when you also use the ajax feature; Select2 will use the value of the data property rather than the data retrieved via AJAX to populate the dropdown, which most likely isn’t what you want.

However, with 22K devices and the way you’re handling them now, your Select2 is going to be extremely slow. (In my experience, it slows down significantly after about 1,000 items). When using the AJAX feature, Select2 expects the AJAX endpoint (ajax.url property) to filter the results it sends back. At the very least (i.e., without a query term—as in when the user simply clicks in the text box but hasn’t typed anything), it expects that only one pageful of results will be returned at a time. It definitely doesn’t expect 22K results in a single page.

From reading your code it looks like you’ve selected a page size of 30 results per page. Therefore, your AJAX endpoint should return no more than 30 results at a time. If there are more than 30 results for the current query, then the endpoint should also return an indication that there are more results available. As the user paginates (scrolls down or up in the results list), Select2 will send additional queries to the AJAX endpoint with the same query term but an updated page value. It expects the AJAX endpoint will have logic to be able to return the requested page of results.

Since your AJAX endpoint is just a data file, this isn’t going to work the way Select2 expects. You need something like a database server (or a similar application) that can (1) filter your data set based on the user’s query term, and (2) return a specific pageful of results at a time.

If you can’t make that happen, then my recommendation would be:

  1. Load your data into a JavaScript object of the required format outside of the Select2 widget (e.g., using jQuery’s getJson() method) and then supply that object as the value of the data property in your Select2 initialization object.
  2. Set minimumInputLength to a value greater than 0, so Select2 will only render data in the dropdown after the user types a few characters. This will limit the number of results that Select2 will render in the dropdown. However, given the size of your data set, I still expect this could be quite a large amount of data. Note that you cannot use the pagination feature with a local data set (which is what you’ve got when you use the data property), so Select2 will try to render every record from the data that matches the user’s current query [*see below]. This will still take a long time, because Select2 must test the user’s query term against every record in the data set to find the matches.

Given the large size of your data set, I strongly suggest that—if it’s possible—you subdivide your data into smaller chunks, ideally of no more than about 1000 records each. Require the user to select a “chunk” first (however you choose to do that), and then initialize the Select2 with just that chunk. Note that if the user then chooses a different chunk, you will have to re-initialize the Select2 with the new chunk (and I suggest that you destroy the Select2 before re-initializing it, to help avoid memory leaks).

*Note: If you can’t subdivide your data into separate files, Select2 will try to render every record that matches the user’s query. To do this it has to compare the user’s query to each one of the 22K results, and it does that with a linear search (i.e., it compares every single record to the query term to decide which ones to show). As I noted above, if there are more than about 1000 total records, Select2 slows down noticeably. With 22K records, the response time will probably be unacceptable to your users.