Handle a large data set?

I’m a student python/django developer whom is new to select2 4 rc and javascript. I’ve managed to get select2 working properly. However, I have a large list of data that select2 bogs down on. I’ve attempted to use the minimumInputLength, which works fine but the application still bogs down because the amount of data being loaded. The javascript I’m using is below. How can I make initial data load of the template not search data until the minimumInputLength is met?

$(document).ready(function () {
    $('.js-example-basic-multiple').select2({
      minimumInputLength: 3,
      placeholder: 'Select a 3-4 User ID',
    });

  });

I’m rendering the select two function in my html below:

           <div class="col"><h4 style="margin-top: 0"><strong>3-4 ID User Search</strong></h4><select class="js-example-basic-multiple"  value = "{{ userid }}" style="width: 1110px" required>
           {% for user in userid %}
           <option value="{{ user }}"> {{ user }}</option>
           {% endfor %}
         </select>

I’ve tried the solution below that I found on stack overflow, but i believe that requires server side pagination, and using this solution my select2 never occurs and it shows the normal select box.

$(document).ready(function () {
    $('.js-example-basic-multiple').select2({
      minimumInputLength: 3,
      allowClear: true,
      placeholder: {
        id: -1,
        text: 'Enter the 3-4 user id.',
      },
      ajax: {
        type: 'POST',
        url: '',
        contentType: 'application/json; charset=utf-8',
        async: false,
        dataType: 'json',
        data: function (params) {
            return "{'searchFilter':'" + (params.term || '') + "','searchPage':'" + (params.page || 1) + "'}";
          },

        processResults: function (res, params) {
            var jsonData = JSON.parse(res.d);
            params.page = params.page || 1;
            var data = { more: (jsonData[0] != undefined ? jsonData[0].MoreStatus : false), results: [] }, i;
            for (i = 0; i < jsonData.length; i++) {
              data.results.push({ id: jsonData[i].ID, text: jsonData[i].Value });
            }

            return {
              results: data.results,
              pagination: { more: data.more,
              }
            };
          }
      }
    });

  });

There is also this example, but I can’t get it to work and the fiddle.js doesn’t seem to work either. http://jsfiddle.net/wphqwvLf/50/

$(function () {
    items = []
    for (var i = 0; i < 1000; i++) {
        items.push({ id: i, text : "item " + i})
    }

    pageSize = 50

    $.fn.select2.amd.require(["select2/data/array", "select2/utils"],

    function (ArrayData, Utils) {
        function CustomData($element, options) {
            CustomData.__super__.constructor.call(this, $element, options);
        }
        Utils.Extend(CustomData, ArrayData);

        CustomData.prototype.query = function (params, callback) {
            if (!("page" in params)) {
                params.page = 1;
            }
            var data = {};
            data.results = items.slice((params.page - 1) * pageSize, params.page * pageSize);
            data.pagination = {};
            data.pagination.more = params.page * pageSize < items.length;
            callback(data);
        };

        $(document).ready(function () {
            $("select").select2({
                ajax: {},
                dataAdapter: CustomData
            });
        });
    })
});

Your fiddle isn’t working because it’s getting a 404 when it tries to load Select2 resources. From the console:

Failed to load resource: the server responded with a status of 404 ()
select2.github.io/dist/css/select2.min.css
Failed to load resource: the server responded with a status of 404 ()
select2.full.js

Also note that an id value of 0 is not allowed in the Select2 data structure, per the documentation: https://select2.org/data-sources/formats#automatic-string-casting

1 Like

Thank you John,

I was able to get the fiddle example to work by adding:

  <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.js" type="text/javascript"></script>
   <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.6-rc.0/js/select2.min.js"></script>
   <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.6-rc.0/css/select2.min.css" rel="stylesheet" />

Now instead of adding a sequential list I want to pull the field userid that is defined below in my view in django:

userid = ADMirror.objects.filter(student_status_id = 1).values(‘studentntname’).values_list(‘studentntname’, flat=True)

I created a new fiddle.js to do what i’m looking for above, but my problem still exists the entire data set is being loaded when you navigate to the page.

http://jsfiddle.net/jgf5fkfL/66/

Using this fiddle it appears all the data doens’t load until the minimumlengthinput of 3 is met, but the search results don’t return a value. I believe i’m close, but something is missing.

http://jsfiddle.net/jgf5fkfL/78/

I wonder if the entire data set being loaded in your first fiddle is because you’ve created the data within the HTML <select>. Try putting it in a JavaScript array instead and see if the pagination works then. (I’m guessing that since the <option> elements exist, when you initialize the Select2 widget, even though you’ve supplied a DataAdapter, it’s ignored because the <option> elements are there already.)

In your second fiddle, since you’ve specified an empty URL for your AJAX request, the POST request is actually getting made to this URL: http://fiddle.jshell.net/jgf5fkfL/78/show/ (i.e., the URL of your fiddle). That URL returns a 403 (Forbidden) due to the browser’s security model, and that’s why you’re not getting any results. Do you actually have a remote data source (i.e., a server)? You can’t fake that by trying to call your fiddle’s URL (which is what a blank URL value will do), and you can’t use the AJAX (remote data source) feature without an actual remote data source.

Hi John,

Once again thank you for your reponse. I’ve abandoned the second fiddle and been focusing on getting the first one to work with modifications. I’m a bit confused as to how to put my data in a javascript array instead of the HTML select tag. Below is my updated javascript. Currently my database is on a remote server, during development i’m using localhost, but the application will be moved to a server. So I believe I am using a remote data source. The fiddle is just an example of the code.

$.fn.select2.amd.require(
['select2/data/array', 'select2/utils'],
function (ArrayData, Utils) {
  function CustomData($element, options) {
    CustomData.__super__.constructor.call(this, $element, options);
  }

  function contains(str1, str2) {
    return new RegExp(str2, "i").test(str1);
  }

  Utils.Extend(CustomData, ArrayData);
  CustomData.prototype.query = function (params, callback) {
    if (!("page" in params)) {
      params.page = 1;
    }

    var pageSize = 50;
    var results = this.$element.children().map(function (i, elem) {
      if (contains(elem.innerText, params.term)) {
        return {
          id: [elem.innerText, i].join(""),
          text: elem.innerText
        };
      }
    });

    callback({
      results: results.slice((params.page - 1) * pageSize, params.page * pageSize),
      pagination: {
        more: results.length >= params.page * pageSize
      }
    });
  };

  $(".selectuserlist").select2({
    ajax: {},
    allowClear: true,
    width: "element",
    multiple: true,
    dataAdapter: CustomData
  });
});

The updated fiddle is here: http://jsfiddle.net/jgf5fkfL/105/

My Current HTML is the following:

  {% block content %}
  <form action = "{% url 'submitted' %}" form method = "POST">
      {% csrf_token %}
  {% block extra_js %}
      {{ block.super }}
      {{ form.media }}

      <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.js" type="text/javascript"></script>


        <script src="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.6-rc.0/js/select2.min.js"></script>
        <link href="https://cdnjs.cloudflare.com/ajax/libs/select2/4.0.6-rc.0/css/select2.min.css" rel="stylesheet" />
  <script src= "{% static '/search/user_select2.js' %}" type="text/javascript"></script>

               <div class="col"><h4 style="margin-top: 0"><strong>3-4 ID User Search</strong></h4><select class ="selectuserlist" multiple="multiple" value = "{{ userid }}" style="width: 1110px" required>
               {% for user in userid %}
               <option value = "{{ user }}"> {{ user }} </option>
               {% endfor %}
             </select>

Unfortunately I don’t have any actual experience with the DataAdapter pattern. However, from what I can tell by looking at the Select2 Adapter documentation, you haven’t implemented your DataAdapter correctly. You must implement all the required methods of both the generic Adapter interface and the DataAdapter interface. (However, I’m not sure why you’re providing your own DataAdapter anyway, unless it’s to try to mock a remote data source, but as I note below I don’t think that will work.)

I guess you’re using “local” data in your fiddle for development/testing purposes, but I’m not sure that’s valid for your case, since you’re trying to use features (pagination, minimumInputLength) that only apply to actual remote data sources (i.e., retrieved via AJAX, which you cannot mock up locally in JavaScript).

You mentioned that you already have the remote data source, so why not just hook up your fiddle to the actual data source?

To be honest John, i’m a bit confused as to what i’m doing with the DataAdapter. All I want to do is load a large amount of data into the select2 multiselect control without it freezing up. I can’t load the data into my fiddle because it’s not data that I want available on the web.

So I think we’re actually back to the original solution you posted, and you just need to figure out how to get that working. I don’t actually have working experience with the built-in AJAX capability of Select2. The one project I’ve used Select2 and remote data with, I loaded the data myself using AJAX outside of Select2, (there are currently a bit less than 400 records, which I process and load into a local array, which is then passed to the Select2 widget).

I would suggest that you create a smaller remote data source that you can use to explore the AJAX capabilities of Select2. Start simple (i.e., just loading the remote data set via the AJAX capability, without any additional features). Then work forward, implementing each feature you need (for example, minimumInputLength), one at a time. Keep in mind that your remote data source will need to accept Select2’s query parameters (term/q, type and page, although you can override these names) and return the appropriate subset of your data that match those parameters’ values.

For instance, when the user simply opens the Select2, it will send a request to your remote data source URL with the page parameter set to “1” and the type parameter set to “query”. Your remote data source should return the first “pageful” of data (however you define one “pageful”–maybe 10 or 20 items). If the user pages down, Select2 will send another request to your remote data source, this time with ?page=2&type=query_append. Your data source should respond by sending the second “pageful” of data.

if the user types a query term (say, “abc”) into the Select2, then it will send a request to the server as each character is added to the query. (I would recommend setting the minimumInputLength and delay options so that Select2 only sends a request after the specified number of characters have been typed and the user has paused typing for the specified interval.) In this case, the initial request to your data source URL will include a query string like ?term=abc&page=1&type=query&q=abc. (It’s not clear why Select2 repeats the user’s input in both the term and q parameters, so feel free to ignore one of them.) In this case your data source would need to return the first “pageful” of data that “match” the search term “abc” (however you define “matching”). Subsequent pagination using the same search term would send new requests incrementing the page parameter and changing type to “query_append”. Depending on how your underlying data source is implemented it might be able to handle the pagination natively. For example, most SQL databases can return data in “chunks” (i.e, “pagefuls”) defined by the query. If your data store is not that sophisticated (perhaps you’re reading the data from the file system instead) then you will have to implement the pagination logic yourself in your data source.

I hope this helps and it’s not too complicated to understand. As I said, start simple and small: get the basic functionality to work, and then work on adding features such as pagination support and search term support.